Scoresheet TTC 2016 Class Responsibility Assignment Case (Responses)
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

View only
 
ABCDEKLMNOPQRSTUVWXYZAA
1
TimestampWhat is your name?
With which programming and transformation languages have you implemented model transformations before?
With which other programming languages are you experienced?
Which solution are you reviewing?
What did you like about the solution?
What did you dislike about the solution?
Do you have any other comments about the solution?
Do you have any feedback for the solution submitters to improve their solution or paper?
2
5/24/2016 15:09:55
Kristopher Born, Stefan Schulz, Daniel Strüber, Stefan John
Henshin, Xtend
Java, Scala, PHP, JavaScript, C, C++, Python
NMF2.4
- Compact specification, enabled by a profound analysis of the CRA formula.
- The robustness of the solutions results. The results are always the same, while the results of other solutions vary with each run.
The underlying hypothesis that genetic search algorithms are not suited for the case due to runtime behaviour is interesting. Anyway the results of the other solutions in terms of runtime and CRA values reject the hypothesis. Maybe even larger input models and/or improvements of the incremental mode could support the hypothesis.
The idea for further improvements involving incrementality and balanced search trees is interesting and the results might be exciting.
-
3
5/24/2016 15:14:47
Kristopher Born, Stefan Schulz, Daniel Strüber, Stefan John
Henshin, Xtend
Java, Scala, PHP, JavaScript, C, C++, Python
VIATRA4
- Impressive presentation of the VIATRA-DSE framework and its functionalities, like multi objective, state encoding, and termination criteria. The solution indicates that this framework is highly suitable to address the CRA case.
- The authors performed an additional experiment with an alternative solution based on a single rule that joins classes.
- Best solution in terms of the achieved CRA values.
The high quality of the results is traded off for long runtimes. Based on the best achieved results, the runtime seems to be acceptable, even though they seem to be a little bit high compared to other similar solutions.
How is the trajectory recorded when a crossover is performed? In the case of a crossover, two different trajectory contribute to the trajectory of the crossover result.
We observed in our own solution that the influence of our crossover operation is negligible. Maybe the authors could do some measurements to quantify the influence of omitting the crossover in regard to runtime and achieved CRA values.
4
5/24/2016 16:05:55Christoph Eickhoffjavac, c++ATL3.2
The performance is high and the atl code is not that difficult to understand.
The bad CRA resukts of the input c,d,e
5
5/24/2016 17:32:16Maximiliano VelaATL, Henshin
PHP, Java, C, COBOL, Prolog, Javascript, Visual Basic
Henshin4
They get pretty good results in considerably little time
It took us time to evaluate the correctness and performance since it was calculated for each individual strategy and not for the overall execution
6
5/24/2016 18:05:14Maximiliano VelaATL, Henshin
PHP, C, Java, Prolog, Visual Basic, COBOL
SBMLib1.6
The paper, strategy and results were really good.
The SHARE demo was presented poorly, and some details in the paper didn't look completely right (i.e. execution times).
7
5/24/2016 19:59:58Albert Zündorf
Fujaba, SDMLib, Progres
Java, Modula3, Modula2, Pascal, Ada, Eiffel, C++, C
MDEOptimiser4
Genetic algorithm, easily configured
They did not manage to apply Henshin rules on different matches
Using Henshin rules but not beeing able to apply them to all possilbe matches is a major drawback of this solution. Otherwise that would have achieved much better results.
You need to ask the Henshin people how to apply the rules to all possible matches.
8
5/25/2016 14:23:05Georg Hinkel
C#, NTL, ATL, QVT-O, QVT-R
Python, Java, SQL, C++, VB.NET
SBMLib2.4
The usage of search-space exploration is a good added value that model transformation engines may offer.
The paper has serious issues, specifically regarding the presentation of results.
I could not find the implementation of the exploration strategies. Have they been added to SDMLib specifically for the TTC? The solution should be clear about that.
The performance figures should be logarithmic, otherwise one cannot see anything for the smaller models. The paper should be clear on what exactly is measured. The conclusion section should not be empty.
9
5/25/2016 14:31:30Georg Hinkel
C#, NTL, ATL, QVT-O, QVT-R, T4
Python, Java, SQL, VB.NET, VB, C++
SIGMA4
The usage of MOEA yields very good results for input models A-C and OK results for models D and E.
The configuration for the algorithms is based on educated guess (namely from the referenced paper) and is not adapted in each case. For larger models, this leads to not so good results.
10
5/25/2016 15:14:53András Szabolcs NagyJava, Xtend, VIATRAC#NMF4
I liked the heuristic used for the transformation.
The author mentions NMF Expressions several times and that it supports incrementality, but for me it was unclear what that really means.
11
5/25/2016 16:04:16Leif Arne JohnsenATL and Groove
Java, C#, C++ and less experienced with haskell, prolog and scala
VIATRA4
Very flexible with multi objective functions. They also seemed to have the best CRA values of all the provided solutions.
This solution is really solid, so it's not much bad to say. Performance was the only negative, but with their approach you're almost certainly bound to get a worse performance.
12
5/25/2016 16:11:37Leif Arne JohnsenATL and GrooveC++, C# and Java. MDEOptimiser2.4
The effort put into creating their own DSL for optimization problems.
Unable to find optimal results, because henshin is unable to apply rules to all possible matches, resulting in a loss of search space.
13
5/25/2016 18:59:59András Szabolcs NagyJava, Xtend, VIATRAC#UML-RSDS2.4
The level that the genetic algorithm is configurable.
The presentation is made very poorly.
If you put a piece of code into a paper, please separate it from the text and explain it in detail. Those expressions are especially hard to read. The figures also needs to be explained more clearly.
14
5/25/2016 22:26:47Filip KrikavaATL, ETL, SIGMA
Java, Scala, Smalltalk, Python, C, ObjectiveC
ATL3
It is an interesting idea to use the simulated annealing algorithm. It easy to understand, easy to implement, yet the results are not the best (comparing with the other competitors).
There is too many tuning variables. Both the report and the solution should really be polished quite a bit.
Provided on github issue #21
It would be interesting to compare with a version without ATL, with just plain Java.
15
5/25/2016 23:35:00Filip KrikavaATL, ETL, SIGMA
Java, Scala, Smalltalk, Python, C, ObjectiveC
UML-RSDS3.2
It was interesting to see how a GA is implemented in UML-RSDS. The execution time and the results are good.
I could not really see the tool itself and it is clear how much was it done in the tool and how much in Java itself. The report is a bit hard to follow wrt the code listings.
I would be interested to see how much gain is in the preprocessing phase.
Details in github issue #22
16
5/26/2016 0:42:49Alex BurduselJava, Xtend
Java, C#, Scala, JS, PHP, Go
SIGMA2.4
The solution was easy to run and it has used genetic algorithms from the MOEA framework. It was easy to run both locally and in the SHARE environment.
The custom model encoding makes the solution very problem specific and if there are any requirements changes, the suggested encoding may not be complex enough to support additional constraints.
Please refer to the github issue for more comments.
17
5/26/2016 0:44:35Alex BurduselJava, Xtend
Java, C#, Scala, JS, PHP, Go
Excel2.4
The solution is very fast and the obtained CRA is quite good in comparison with the other solutions.
Although it has a friendly GUI the solution is platform dependent. I could not run it on a Mac or Linux machine, but it worked well in the SHARE environment.
Please refer to the github issue for more comments.
18
5/26/2016 16:05:33Kevin Lano
ATL, QVT-R, UML-RSDS, Java
C#, C++, CExcel3.2
This has clearly produced good results, and the efficiency is impressive.
Not clear how it uses transformation technologies, instead of existing non-MT optimisation tools.

In addition, the heuristics in Section 3 seem to require tuning of the algorithm to specific CRA problems. The idea of creating a main class could lead to unbalanced class diagrams. Such diagrams are local but not global optima. The approach does not seem to produce optimal solutions (eg., for C, D, E cases).

19
5/26/2016 16:27:25K. Lano
ATL, QVT-R, UML-RSDS, Java
C++, C#, CHenshin3.2
An interesting investigation of GA application to the problem.
It is unclear how Henshin rules are integrated into the GA - is the control of these external?
Is the GA and rule control coded in Java? Some example code would be helpful.

The population sizes used seem quite small.
Section 4: 'CPA' should be 'CRA'

At some point the best results obtained per model should be given.

Section 4.2: explain the abbreviations RS, JSC, etc.
20
5/27/2016 15:47:39
Antonio Garcia-Dominguez
Java, XSLT, Epsilon languages
C/C++, PythonExcel2.4
It produces good results in very little time.
It does not actually do search-based model transformation "on the fly", as one would expect from the case description. Instead, it transforms the model to a representation and then uses a greedy approach to gradually build up a single solution.
There was a bug in the provided VBA code (fix is mentioned in the Github issue).
Since the approach appears to be quite efficient, it might make sense to spend extra time in keeping the N best solutions at each step instead of keeping just one. That way the user would have a knob to tell the algorithm how hard it should work to obtain the best option.
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...