Transformation Tool Contest 2017 Open Peer Review State Elimination (Responses)
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

 
View only
 
 
ABCDEFGHIJKLMNOPQR
1
TimestampWhat is your name?
Which solution are you reviewing?
Does the solution implement any of the extension tasks?
How do you rate the overall quality of the solution?
Is the solution correct?
How do you rate the suitability of the tool for state elimination?
How confident are you about your evaluation?
What did you like about the solution?
What did you dislike about the solution?
Do you have other comments about the solution?
Do you have any feedback for the solution submitters to improve their solution or paper?
2
6/8/2017 22:02:12Georg HinkelSDMLibExtension 13No42
The rule visualization of SDMLib makes it really understandable what the transformation does at a high level. The performance comparison with the reference solution looks good.
It is hard to judge the solution entirely without taking a look at the entire source.
I would enjoy more details on the evaluation. In particular, how many lines of code does your solution have?
I would like to see 1. a source code repository, 2. instructions how to run the solution and 3. an evaluation with regard to non performance-related properties such as lines of code of the solution.
3
6/10/2017 8:27:45Georg HinkelHenshin
Extension 1, Extension 2
4Yes44
The graphical programming approach is very interesting and in many cases, it is very understandable what the tool does precisely, which is quite cool.
I followed exactly the installation steps from the Github repo but still got errors opening the transformations. There are some units that are very similar, such as fixTransitionsUnlooped and fixTransitionsLooped. It would be good if Henshin could be configured to express that the match of the self-transition is optional.
As far as I understood, the solution currently eliminates states in a random order. You might improve your performance results if you influence the order to eliminate states ordered by the product of incoming and outgoing transitions. I would be very interested to see how this is possible with Henshin.
The improvements of the language suggested in the paper are very interesting. Probably you may want to remove references to github issues, it sounds a bit strange.
4
6/11/2017 19:27:48
Mohammadreza Sharbaf
NMFExtension 13Yes35
It produces results in very little time.
The solution uses a code centric strategy and need to transform the Ecore metamodel to the NMeta format to load the input model.
Please refer to the github issue for more comments.
Please refer to the github issue for more comments.
5
6/11/2017 19:31:11
Mohammadreza Sharbaf
SDMLibExtension 11Yes25
It is an interesting idea to use pattern object for matching.
The presentation is made very poorly.
Please refer to the github issue for more comments.
Please refer to the github issue for more comments.
6
6/12/2017 18:23:37Daniel StrüberNMFExtension 14Yes44
Following some communication with the case author, I was able to run the solution and reproduce the excellent execution times as indicated in the paper. I obtained correct solutions rapidly for all models except for leader6_5, leader6_6 and leader6_8.

Despite being based on in general-purpose-code, I think the solution is largely adequate for the problem. The involved graph structures are relatively simple, and so it's fair to expect that developers can understand the transformation by looking at the code. I mostly found the listings readable, except for some NMF-specific terms, like "FirstOrDefault". These should be better explained in the text.

The performance evaluation only considers the transformation time, whereas the evaluation framework by the case authors takes into account the reg-exp evaluation as well. This should be pointed out in the paper, and considered during the final performance evaluation (if there's any).
The paper seems to assume that "concise" generally implies "readable" or "high abstraction level". Even though I find the particular solution readable, I find this more general assumption somewhat debatable (the code golf community may disagree).


It would be very interesting to learn more about the reason for the sudden performance decline for larger models. As pointed out by the author, the abrupt leap from a very fast solution to a non-working one is a bit puzzling.

7
6/14/2017 0:12:10Daniel StrüberYageExtension 13Yes42
The way I understand the solution, it is based on a number of rules for dealing with specific cases of eliminating a state, and a control flow for orchestrating the rules, leading to a natural and generally suitable solution to the case.
An aspect of the solution description I found hard to understand is where the eight different cases for "rerouting edges" come from, which lead to a total of 20 rules. Judging from Listing 3, the case distinction with manifold patterns in the style of "pkKpkK" may be unneccesarily complicated.
"Does the solution implement any of the extension tasks?" The answer is actually no for both cases, but I had to choose at least one item.

In addition, I chose a low confidence value because I was not able to verify the performance results.
The paper sells Yage as a "fresh start", focusing on "simple data structures and algorithms". However, from the presentation it's not so evident what this simplicity entails. The idea to specify rules as graph patterns where elements have different actions (visualized as colors) is just like in other graph-based transformation languages. While I do not expect a related work discussion, it might be better to be more specific when claiming novelty.

I didn't understand the mentioned issue with the naming of edges. Representing a name in an object-structure is simple, so when aiming a deliberately simple design, I wonder how one winds up with a complicated solution.

The depiction of the rule in Figure 2 has some formatting issues, like illegible labels and labels overlapping with arrowheads.
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...
 
 
 
Form Responses 1