Demon Castle Tutorials - Usability Test [Was: Re: [Etoys] Demon
Castle Tutorials for eToys]
milan.zimmermann at sympatico.ca
Mon Oct 23 02:42:34 EDT 2006
This weekend, we ran a usability test on your tutorial. The way I setup the
environment is I asked 3 girls (2 of my daughters, both age 15, and their
friend, same age) to do, separately:
- Start each of the 3 projects of the tutorial (I did the start/stop)
- Be in a room with a computer by herself, with no help
- Call me in if she gets stuck, BUT only after 2-3 minutes of trying. I
helped only the minimum necessary in such case.
All was repeated with each. All 3 finished all 3 tutorials, some help was
needed, Jacqueline only needed help once, to "repair the script".
I realize the ages are probably older then the target age. Also my girls did
use eToys quite a bit before (up to about 2 years ago, when they used it
last .. I noticed they did a _very_ good job forgetting it :) ). Their friend
never used eToys; but all used computers to a fairly large degree.
I basically decided to note (and report here) things on which they get stuck
too long or surprised by. The report below is sort of a summary of the 3 of
them. First, let me say that you would be happy to see (and hear) their
- Wow, this is pretty cool
- Ah, I can use it
- What i cute demon :)
Please do not take the length of this "report" as a negative sign - the girls
did love the tutorial, and all got to the end with some help, Jacqueline
almost did it all by herself!
So the report:
A. Generic Findings:
- Kids are impatient :) they cannot wait 2 minutes without asking for help,
but get better over time.
- When they get lost, they make things worse by just clicking around, often
trying things they learned in previous lessons.
- The "Try Again" button was valuable, but it would be nice to add a note
(bold) "You can always click on the "Try Again" button". Also is it possible
for the "Try Again" to completely restore the state, including deleting
copies, restoring destroyed scripts etc? For example, sometimes they ended up
with many copies of the key (as they decided to try the "duplicate"), and
things became confusing. Also, Joelle deleted the scriptor, and that was the
end of her lesson until I restarted it. So restoring the page state
completely on "Try Again" would be great.
- We had a few MNU, but they did figure to just click the "x" on the left,
and kept playing. This is funny, for a developer it is a mojor event. For a
child, as long as they know what to do, just get rid of it and keep going...
- Not sure how hard this is - is it possible to "lock up" the instructions,
in the sense kids cannot write to it, a few times they got confused and
started dragging out images from the instructions
B1. Handles/Hallo Tutorial:
- Clicking on the black parts of the "key morph" does not act on in. I
realize this is etoys behaviour, but it confused one of them, brought
confusion about how the halo is brought up, and I had to explain. I think for
the initial tutorial, the key morph should have a black background, so it is
sensitive to click anywhere on it or close to it.
- Apart from that, pretty smooth sailing!
B2. Paint Tool Tutorial:
- I think the "auto-painter" confused more then helped, overall. I would
probably remove it, caused much confusion for everyone. (Also, once painted,
the painted morph stays around on all following pages it seems?).
- On the following few "Paint my face" instruction pages: In the
instructions, perhaps instead of pasting the icons, better would be to start
with hallo up, and put the "Paint Box" close, or inside, of the
instructions?. The girls thought that they should click on the icons inside
the instructions, instead on the PaintBox all the way right. Eventually they
ploughed through it though :)
B3. Scripting Tutorial (hardest, understandably, but excellent):
- a movie would be ideal (Broken scriptor: hardest to get right)
- The "Try Again" comment from A. Generic Findings: applies.
- The girl who did not use eToys, was confused and kept asking "what is
scriptor"?, "what is a tile"? - We should define and
describe "scriptor", "tile", "exclamation", and "clock", in more detail,
perhaps a more text and arrows would help. It is hard to describe though...
- The arrow "Try the exclamation button" should point to "!" (points to
- Page "Is the scriptor broken" page: Perhaps could add more description
like: "Once the scriptor is repaired, click the clock as in the previous
page". This is where the girl who did not use eToys started to get stuck.
- Page "Let me explain a bit about a scriptor here.": - What do you think
about moving this page up front, maybe simplifying it (as noted 3 items
5. At the end:
This was funny, where the demon says "I am sorry I captured you. But I just
want a human friend. I am a lonely demon.", the girls all noted "why does he
ask about a human friend, it should be a cat friend?" I never noticed it at
Beautiful stuff - please let me know if we can help. We noticed a few typos
and formulation potential changes - if you'd like, we can summarize them.
Thanks Milan, Jacqueline, Joelle and Najiba
On 2006 October 20 11:08, Takashi Yamamiya wrote:
> Hi Andreas, Milan,
> I have updated handles tutorial.
> Milan Zimmermann wrote:
> > Takashi,
> > I remember one thing I got stuck on (almost) completely, was when I
> > started resizing the "BIG KEY". When you try to make it big and small
> > again in a few directions without letting go (and it goes
> > LARGE/Small/OK"), it becomes almost impossible to hit the OK size. Maybe
> > it should be more generous with the range of OK (it seems to use a
> > relative measure perhaps). I will have more notes over the weekend,
> Aha! everybody who tried it complained this issue, but I just forgot...
> I made it easier, thank you!
> > On 2006 October 19 23:58, Andreas Raab wrote:
> >>> http://tinlizzie.org/olpc/tutorial/DemonCastle1.004.pr (handles)
> >> * On the first page where it says "Click THIS arrow" I'd move the text
> >> closer to the arrow. I was kinda looking for "another" arrow because the
> >> one you mean is quite far away from the text.
> Oops, you are right. That was bad.
> >> * On the third page (with the first key) I would change the text from
> >> "Go ahead you can move the key" to something like "Go ahead, drag the
> >> key on the keyhole" (to emphasize that you mean click and drag)
> Make sense,
> >> * On the "resize handle" page it was very hard to find quite the right
> >> size for the key. It seems as if there is only a small range in which it
> >> will fit and it was quite tricky to get that "just right". I wonder if
> >> there is an easy way to increase that range?
> I am going to fix other two projects later.
> Thank you so much!
More information about the etoys-dev