ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
Redo reading_nearby_exploreNew Kantor
2
Item1st versionRevisedDone
3
check the cloud research and Amazon balances and make sure they are adequate for this and other concurrent experiments.
4
make individual rules as needed
5
check the syntax etc. at Validate your experiment plan (wisc.edu)
6
git pull origin master
7
place them in game-data/rules/APP
8
git add as needed
9
git commit and write a useful note about what they are
10
git push origin master
11
wait a bit
12
test them in APPS http://sapir.psych.wisc.edu:7150/w2020-dev//launch/launch-rules.jsp
13
revise as needed. When the rules are OK, place them in
14
game-data/rules/[yourInitials]/experimentName/*
15
make trial files in game-data/trial-lists; suggest: /fl/experimentName/*.csv
16
git pull origin master
17
git add as needed
18
git commit and write a useful note about what they are
19
git push origin master
20
wait a bit
21
check the entire experiment structure at http://sapir.psych.wisc.edu:7150/w2020-dev/check-plan-form.jsp This will indentify any typos in the trial*.csv files and catch those errors, which may not be visible when you test the experiment, as you may not get the bad trial.csv file.
22
check the experiment at the trial level by playing the HTML test server, ensuring that the pieces that should be moveable are indeed moveable, that all pieces are appearing as expected, and that the structure of the rule is appearing as desired: http://sapir.psych.wisc.edu:7150/w2020-dev/html-play.jsp
23
test in developer interface Play using the GUI front end (wisc.edu)
24
look at prod interface Play using the GUI front end (wisc.edu) Use "show introduction to make sure you have the one you want.
25
revise as needed. A git push will overwrite the bad files.
26
semantic check: when the rules and trial files look correct (try to get one other person to check them)
27
set up the CloudResearch for a pilot CloudResearch | Online Participant Recruitment for Research & Surveys
28
run the pilot. Monitor your email for problems.
29
check in /opt/tomcat/guesses to see how many subjects you've got (command such as ls -alt | more don't data extract until you have enough
30
do the extraction The Analyze Transcripts tool (wisc.edu)
31
look at the data [here there are some tools of Paul’s and some scripts that Gary has]
32
compute the bonus payments as shareOfTotalPoints * (totalToSpend) See this sheet for how to do it.
NOTE: Paul and Eric used the principle of (a) only bonus the top half of the performers in terms of some score. (b) divide the bonus among them in proportion to their scores, holding the average at $1 per participant.
33
do the "bonus payments process" CloudResearch | Online Participant Recruitment for Research & Surveys
34
then do the large data run. To do this, use CloudResearch; make a copy of the study, with the needed number of additional subjects. Then go to dashboard and launch it. Monitor your email for problems. . When data collection is complete (usually just a few hours with CloudResearch subjects)
35
1. do the extraction
36
2. look at the data and guesses
37
figure out who is "not approved" and enter them at
https://app.cloudresearch.com/Worker/ApproveAndReject
38
3. do the "bonus payments process"
39
Eric has developed a bonus computation process.
1. rank all the players' scores and find the midpoint.
2. those below midpoint get no bonus
3. identify the median score and the maximum score
4. map each player in the top half to the range [0,1] corresponding to their position in [median score, top score]
5. the players receive their score in [0,1] in dollars
40
4. do careful (statistical) analysis. Identify any statistically significant differences in ruleSet difficulty. Look for training effects.
41
5. write up your conclusions about this run, its own parts, and relation to any others. A writeup should be a document with explanations and citations. Not just a statistical output.
42
6. ask someone else to read your writeup, while you still remember what you did
43
7. revise and clarify as needed. Report at a group meeting
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100