How can I turn this into a formal piece of research?
The key question:
Are internet communication tools (skype + Talk&Write) is potentially an effective ICT tool to use when extending students oral language and writing?
Group A
This group were engaged in the use of skype and talk&write technology to communicate with a mentor ( who were 3 years older than themselves)
Group B
This group of students worked with a ‘talking buddy’ in class to edit their writing.
There was no teacher content input for either group at this part of the writing programme. The teacher demonstrated the original writing piece as the students prepared their first draft.
At the end of the trial ‘hardcopies’ were compared and recorded conversations were reviewed.
Background
In his book ‘Computers as Mindtools’ Jonassen augues that tools such as Skype and Talk and write can support networked learning ciommunities with other students, mentors, experts or others. Which may enhance their learning. He talks about the social dynamic not as a distraction to learning but as being critical to the learning process.
Through opportunities presented by skype and its plug-in students are exposed to a greater diversity of experience and persepectives.
Conversation over the internet has the same potential for ‘chatting’ and ‘off-task’ behaviour as any regular classroom context. Acknowledging that there is value in ‘open-conversation’ with fellow students in different geographical locations there still needs ot be focused, purposeful learning.
Talk&Write offers students that. The ‘whiteboard’ feature acts as ‘the object in shared workspace’ which Jonessen states as an essential for increasing the length of productive ‘on-task’ behaviour. This whiteboard “…represents, in itself, the fruits of their intellectual labour.” (p.239)
Some observations
-
It was difficult to monitor those online.
-
The motivation of the group using the technology was visibly higher than Group B.
-
The written record of Group A showed a wider use of vocabluary and greater use of complex sentences than those in Group B.
-
Listening to recordings of students working indicated that Group A were enagaged in ‘ontask’ behaviour for longer than Group B.
-
The technology had a number of issues; sound cutting in and out, skype dropping of on several occasions and the video ’stalling’
Some questions
Though it would be not accurate to say that nothing was learned, it is not possible to draw any confident conclusions.
So:
-
What is needed to move a piece of classroom research, such as the one described above, from informal to formal?
-
How formal does research need to be before it is of use?
-
How do we define ‘of use’?
-
How could this research be conducted with more rigour?
-
What is the place of informal research, such as this, in the classroom?
Reference
Jonassen, D. (2000). What are mindtools? Computers as mindtools for school. Engaging critical thinking (2nd ed.). (pp. 3-20).
Columbus, Ohio, Merrill.
I tried to do something similar with Maths earlier this year and had the same problem http://marniethomas.blogspot.com/2007/07/digis-vs-papers-teaching-experiment.html.
I thought about swapping each group the next day to make the sample fair but then there could be some factor that affects learning that day so still not a fair test. Perhaps it is just a matter of taking lots of samples of different things to compile the evidence? I was going to but it became too time consuming to record and write it all up.
Perhaps using an impartial observer to take the observational notes would work? But then where do you get someone in a school who is impartial to the effects of ICT – they are usually for or against!
I will watch for comments on this post with interest.
I guess you could start by clearly what are the conditions of value when extending students oral language AND what are the conditions of value when extending students writing? They may not necessarily be the same thing.
Your research question could then be framed around one or both of these conditions so that “effective” becomes clearer – and what you are tring to “make better” is clearer
Getting the question right is key
Then look carefully at the kids in your two sample groups – and the intervention you provide – with and without technology Sometimes technological interventions in learning have been shown to adversely affect student learning – cognitive load arguments etc – and effects have also been shown to depend on the ability of the kids in the research group
Then I reckon you’d need to work carefully to control all the other variables – you have a number of significant variables that need controlling in this research which is why it cannot tell you anything and why it cannot be generalisable to other kids
Issues of validity and reliabilty are what is compromising this – plenty of ideas on how to design research around if you want to get on top of this
You are not alone in this struggle to “prove” something – Asher (2003 p.7) warns that “The published literature in the field of gifted and talented education is inherently flawed.” Sample sizes tend to be small, (70 or less); our experimental curriculum reform practices are not “spectacularly better than our current practices” and narrative reviews are “prone to “subjective selection biases” of the reviewer.
I wonder what he would make of the action research projects that abound “proving” the use of ICTs do this that and the other
Hi Simon,
Media differentiation studies are always going to be difficult. There are so many variables (welcome to educational research) that can “cause” a measurable difference and not always the ones that you want to
Think less out media differentiation (Skype + Talk&Write VS face-to-face) and more about the effect that the technology environments might have on collaboration (both in class and beyond). How are they communicating during the sessions and how have those types of interactions been shown to influence learning (match with the literature)? Do the students in the face-to-face groups continue collaborating (or simply meeting) to a greater or lesser extent than those in the Skype group?
In the scenario that you set up here, I would make a few suggestions.
1) Assess all of the students prior to the intervention in a manner similar to the post-intervention assessment (writing samples). You can’t show change unless you have a baseline assessment (this is particularly problematic with non-randomized samples and small sample sizes).
2) Provide paper to the face-to-face groups to use when collaborating. I’m assuming that they sketched out ideas in a way similar to those on Skype. Save those papers and save the work done on the whiteboard (a screen recorder would do this well).
3) Think about do more with your audio recordings. These can be invaluable when looking into processes at work, especially with groups. Look into discourage analysis for analyzing this data. You can likely get a paper out of this
Good luck,
Dan
Hi Simon
One thing I would add to the excellent suggestions above is that you need to clearly define your indicators. You need to know what you are looking for. You have mentioned that motivation and engagement was high, but you stated you were looking at improvements of oral language. There are so many factors in an exciting project as using communication tools that you need to narrow these down. How can you use the improvement in writing as an indicator of an improvement of oral language, you may need to choose one or the other? You need to narrow your project right down to the indicators you have identified. I agree completely with Artichoke, refining your question is the first and highly important step.
The qualitative research you describe is more valuable than quantitative research, because none of us teach in a research laboratory. Chat logs can be accessed for all Skype chats/calls. That might assist with monitoring after the fact, but not during. How did one measure “The motivation of the group using the technology was visibly higher than Group B.” and “The written record of Group A showed a wider use of vocabluary and greater use of complex sentences than those in Group B.”? What were the standards against which these were measured and with which assessments? How were the assessments normed? How long was this study and how were the groups chosen – were they in different schools or varying socio-economic status? And why don’t I just go make coffee and be quiet now?
Haven’t read everyone’s responses just yet so apologies if I duplicate anyone else’s suggestion. If your focus is an improvement in oral literacy you could identify some discourse (certain vocab/phrases) that you would expect to see used – that demos critical analysis etc and then search the logs (chat or oral) for examples – did these language features increase in the course of the activity?