Bringing Citizen Scientists and Historians Together

http://www.oldweather.org | http://www.solarstormwatch.com

Abstract

This paper outlines how citizen science projects Solar Stormwatch and Old Weather play to the potentially different motivations of science and history enthusiasts. It draws on informal feedback from the forums and other social channels but also references well-documented crowdsourcing projects Galaxy Zoo and the Australian Newspapers Digitisation Program. It concludes with recommendations for attracting both lightweight contributions and sustained collaboration in online volunteering projects.

Keywords: citizen science, crowdsourcing, digitisation, communities, science, history

1. Introduction to citizen science

The National Maritime Museum works as part of the Citizen Science Alliance (http://citizensciencealliance.org) to produce crowdsourcing initiatives in order to further science itself, and the public understanding of both science and the scientific process. The museum has been most involved in the development of Solar Stormwatch (http://solarstormwatch.com) and Old Weather (http://oldweather.org). Both of these projects are what the Alliance describes as 'citizen science'; they use the time, abilities and energies of a distributed community to analyse scientific data. This approach to research responds to increasingly vast data sets that both exceed academic capacity and elude automatic classification by computers. Citizen science harnesses the unique capabilities of many human eyes.

The best example of such a project is Galaxy Zoo (http://galaxyzoo.org), which was first launched in July 2007. Galaxy Zoo invites members of the public to morphologically classify images of galaxies. In Galaxy Zoo 1, volunteers determined if a galaxy was spiral or elliptical and, if spiral, the direction of rotation. The project delivered more than 100 million classifications of galaxies, published many peer-reviewed science papers, and produced discoveries that made the news headlines.

Figure 1: BBC News, 'Teacher finds new cosmic object'

Launched in February 2009, Galaxy Zoo 2 responded to the demonstrated capability of the Galaxy Zoo 1 volunteers by seeking more in-depth analysis of 250,000 galaxies from the original survey.

A programme of citizen science allows our museum to link cutting-edge science and issues with our historic purpose and subjects. The Old Weather project also delivers a reusable interface for the distributed transcription of archive materials.

2. Understanding what motivates online volunteers

The motivations of citizen science volunteers in Galaxy Zoo are well understood and have been described in a comprehensive study (Raddick, 2009). Motivations range from an interest in the science, to awe and aesthetic attraction, the thrill of discovery, community and learning.

Table 1: Summary of Galaxy Zoo motivations, adapted from Raddick 2009

The Australian Newspapers Digitisation Program (http://www.nla.gov.au/ndp/) is another well-evaluated online volunteering project. Its purpose is to digitise out-of-copyright Australian newspapers originally published 1803-1954, for a planned output of over 40 million articles. To reach this ambitious target, the National Library of Australia invited members of the public to improve and enhance the quality of OCR data by making text corrections and adding tags via a Web interface.

Project leader Holley (2009) found that volunteers were motivated by "[the] addition of new content, the knowledge they were helping other people as well as themselves, the idea they were helping to record Australian history accurately, helping a good cause, [and] being able to make a small but effective contribution to the big picture."

Understanding the motivations of online volunteers is an important way to inspire and sustain participation. From her interactions with volunteers, Holley concluded that motivation could be increased through "public recognition, public ranking tables, user profiles, and the ability for [volunteers] to communicate with other text correctors."

The Citizen Science Alliance is guided by a philosophy that all projects must answer a real scientific research question. The projects must never waste the 'clicks', or time, of volunteers, who should be respected as collaborators, including, where appropriate, recognition as co-authors of academic papers. Shortly after its launch, Old Weather reduced the minimum number of repeat transcriptions from five to three, because a review of the data demonstrated that just three transcriptions produced reliable results. The Galaxy Zoo project is exploring how the data produced by its volunteers can work as training sets for machine learning, so more routine tasks can be handed back to computers.

3. Solar Stormwatch: playing to motivations

Launched in February 2010, Solar Stormwatch is a Web site that invites members of the public to spot explosions on the Sun and track them across space to Earth, using video data from NASA's twin STEREO spacecraft. Solar storms have the potential to interfere with communication satellites, upset GPS navigation systems and pose a health risk to astronauts on the International Space Station. Stormwatch volunteers mark any visible solar storms in the STEREO videos, and then trace the progress of a storm through composite images to calculate an accurate speed and direction. Working alongside professional scientists, the Solar Stormwatch volunteers contribute to solar research and build an early warning system.

Figure 2: Homepage of the Solar Stormwatch website

At the very start of the project we consulted game designers Six to Start (http://sixtostart.com) on the ways that game mechanics could be used to motivate volunteers. From Raddick's 2009 study of Galaxy Zoo volunteers, the museum had identified two areas for improvement: Galaxy Zoo's collaborators were predominantly male and only a very small percentage was attracted by the social benefits. Six to Start was invited to help shape a more playful, sociable experience of citizen science.

To meet the high standards of the Citizen Science Alliance, every task within Solar Stormwatch needed to have a real research purpose. To meet the museum's aspirations for the project, every task also needed a payoff for the volunteer – from instant feedback to a record of achievements and conversation starters.

Playing up the rewards

Many of the contributors to Galaxy Zoo and the Australian Newspapers Digitisation Program describe the experience as 'addictive' and we wanted to see how far we could push that level of engagement in Solar Stormwatch.

First up, we realised that it was important to work with the near-real-time data that was available, in addition to the video archive. Our volunteers get to see the latest data from the STEREO spacecraft; every hour the spacecraft sends a compressed 'beacon mode' data packet back to Earth. On reaching the receiving antennas of NASA's Deep Space Network, it whizzes through a data pipeline via JPL and Goddard Space Flight Centre before arriving at Rutherford Appleton Laboratory near Oxford in the UK. Here, it's automatically converted from FITS into Flash movies, then sent on to our partners at Oxford University. This data isn't beautiful but it does let our volunteers spot Earth-bound solar storms up to three days before they reach us.

We also wanted to telegraph our big payoffs. For example, since we know that volunteers are motivated by the potential for discovery ('I can look at galaxies that few people have seen before'), we tell Solar Stormwatch participants when they're the first to see new data from space. Informal feedback on social messaging platform Twitter (http://twitter.com) suggests that our volunteers appreciate this feedback: "The best thing is, the site tells me I'm the first to see the data."

We also introduced more explicit achievements that are recorded in badges on the volunteers' profile pages:

• "Whoosh! That's the fastest solar storm spotted so far"
• "Keen eyes. You were the first to spot this solar storm"
• "Earth protector. You spotted an Earth-bound solar storm"
• "Most curious. You reported 10 interesting features."

As a challenge to go further, our volunteers can follow the same storm through a number of data-analysis steps, ultimately tracking it back through all of the instruments onboard the STEREO spacecraft. And every detected storm is published to a page listing all of the people who helped to identify and analyse it. Ultimately, we plan to give these 'spotters' the facility to name their storm, just as hurricanes used to be named by their forecasters.

Personal and collective results on-the-fly

Previous citizen science projects made their result calculations behind the scenes but Solar Stormwatch volunteers get their results – personal and collective – on-the-fly, thanks to a set of carefully-honed algorithms. For example, a volunteer who traces a composite image gets an immediate translation of these clicks into a likely speed and direction for the storm.

The ultimate expression of this approach is our user-generated space weather forecast. We analyse the accumulated results from our volunteers to identify clusters around a point in time. That means members have spotted a solar storm that we can be reasonably sure exists. From this point we use the collective measurements to make another calculation: the storm's speed, direction and likely arrival time at Earth. This gets pinged to Twitter as our space weather forecast (http://twitter.com/#!/solarstormwatch). Our first user-generated solar storm alert was published to our 2,500 Twitter followers on 10 December 2010 and reported in the press, including British tabloid The Sun.

Conversation starters

While it wasn't feasible, at launch, to have threaded conversations on every data object, we developed a 'favourites' feature that enabled volunteers to save any piece of data and share it on social platforms via a short, stable URL. We also embed calls-to-action within the interface. For example, if an analysis points to an incoming solar storm, we suggest that the volunteer recruit friends to help confirm the forecast. Finally, we make sure that our Twitter alerts are 110 characters or less, so there's space for other people to 'retweet' them with a couple of their own words.

Look out! An Earth-directed solar storm is expected at 07:42 on 13 December 2010 GMT
http://solarstormwatch.com/storms/193

4. Old Weather: translating astronomy motivations

Launched in October 2010, Old Weather (http://oldweather.org) is a website that asks the public to help improve reconstructions of past weather and climate across the world by finding and recording historical weather observations in handwritten Royal Navy ship logs. The work will influence future climate model projections and improve the database of weather extremes, risks and impacts.

Figure 3: Homepage of the Old Weather website

The logbooks, which date from 1914 to 1923, contain previously untapped information about the weather, ships' movements and daily events onboard. This data is not available anywhere else, and has the potential to advance scientific understanding of climate variability and change. By taking part in Old Weather, members participate in and gain an understanding of how climate science research is undertaken, data are processed and results used, allowing them to contribute towards important climate research activities.

Figure 4: Global weather reconstructions: a short film introducing the Old Weather project (http://vimeo.com/15388983)

Game-like mechanics

Old Weather further developed the game-like mechanics introduced in Solar Stormwatch. As feedback for the project as a whole, there is a dashboard on the homepage showing the percentage of logs completed, the number of pages done and the number of ships complete. On an individual level, volunteers are promoted to 'captain' of a vessel if they complete the most transcriptions for that ship. The ship's journey, as revealed by coordinates extracted from the log, is traced on a map to reflect progress. This inspires volunteers to set milestones for their work: 'I'll continue until we make port…'

This sense of being on a journey also leads volunteers to identify with the ship whose logs they're transcribing, as revealed by the Old Weather forum thread, 'Has anyone else become Endeared…' (http://forum.oldweather.org/index.php?topic=71.0):

Post by: Mister Meaux on October 15, 2010
I picked the HMS Minotaur almost at random. The name was intriguing. I'd heard of her but knew nothing of her. Now, I'm in it for the entire cruise. There's a lot of work with the picket boats while she's in harbor. There are records of various ships that enter and leave Hong Kong harbor. I know that later she'll be off the Icelandic coast making soundings in a gale and she was at the Battle of Jutland. I almost feel like a part of the crew. Yes, I'd say I find the old girl endearing.

Post by: choirmaster2000 on October 21, 2010
I have totally become enamored of my ship, HMS Juno. The handwriting is sometimes a challenge, and the notes on various 'events' are quite difficult to read. But it's a fascinating glimpse into shipboard life, and you do start to feel a sense of pride in 'your ship'.

Post by: galaxirose on October 22, 2010
I feel the same way about my ship! I'm on the HMS Newcastle and we've been sailing up the coast of Chile for awhile in 1915. I kinda feel like I'm on the boat? Like, oh it's windy today. Oh nice storm!

Re-enacting history through social media

While Old Weather has a clear focus on recording weather observations, the transcription tool allows volunteers to capture other information of historic interest, extending our collective knowledge of Royal Navy history during and after the First World War. By January 2011, the database had additional information from almost 10,000 log pages, representing 126 different ships.

Figure 5: Old Weather transcription tool

The data used in Old Weather is certainly not 'beautiful' in the same way as the deep space images available in Galaxy Zoo. While there may be an initial pleasure in encountering the poetic language of the weather scale, there was a risk that the task of extracting observations would be considered 'dry' or repetitive after a short time. But our volunteers have been inspired by the real human stories that they discover in the logs and they capture weather data in order to follow the stories of vessels and people through to the end. In doing so, they gain an insight into life at sea during this time period. For example, they come to understand that enemy combat was not as common as they might have imagined, and there is much more in the logs to do with the practicalities of living life at sea.

Post by: screamingguns on October 14, 2010:
It's quite a fascinating glimpse into a world most of us would otherwise not have known.Even the simple things like finding out how much stores a ship took aboard at which port, or what the crew got up to on a daily basis.Like in the log book for HMS Kent on one line around October time there's a note that goes 'Boys back to school', which I thought was a rather touching piece of humanity amongst all the measurements and nautical observations.

(Old Weather forum thread, 'Has anyone else become Endeared…' http://forum.oldweather.org/index.php?topic=71.0)

Post by: Veero on November 13, 2010:
I jumped ship and joined the battlecruiser HMS Invincible. In November 1914 they had fun 'crossing the line' when 'H.M. King Neptune held his court and carried out the customary rites.' (My mother had told me about that when she was travelling to North Africa in WW2).

However, on 8th December 1914 they were involved in quite a scrap in the Falklands involving a number of British and German vessels. 2 German ships were sunk including the flagship of German Admiral von Spee. I was pleased and moved to note that they went full speed ahead to pick up survivors. The following day they held a short funeral service for dead German sailors.

At the end of the same day they altered course several times 'for investigating whales.'I expect they needed nature to help them recover from the terrors of their experience.

(Old Weather forum thread, ' Riveting Log Entries", http://forum.oldweather.org/index.php?topic=209.0)

Interesting and amusing anecdotes from the logs are shared in the forums and on other social platforms, leading one Twitter user, @KarenLMasters, to observe that "a lot of ship captains' logs were essentially tweets with geolocation tags." (http://twitter.com/#!/KarenLMasters)

danach on October 27, 2010:
The Ernest Shackleton came aboard my ship and I thought that was pretty neat!

Dread Pirate Roberts on October 29, 2010:
Who nicked my tatties! Log entry from HMS Amethyst off the coast of Uruguay: '8.0 80lbs potatoes stolen during night'

(Old Weather forum thread, ' Riveting Log Entries", http://forum.oldweather.org/index.php?topic=209.0)

Volunteers are also connecting with each other around their ships' encounters, to find out the other half of the story, or warn of interesting events that they may be approaching.

The re-enactment of history that Old Weather affords is similar to The Diary of Samuel Pepys (http://pepysdiary.com), which was created by Phil Gyford (http://www.gyford.com/) in 2003. Pepys' Diary is a presentation of the diaries of 17^th century diarist Samuel Pepys. A new entry written by Pepys has been published each day since 1 January 2003 when the entry relating to 1 January 1660 was published.

Figure 6: The Diary of Samuel Pepys homepage

These daily updates are available via RSS or e-mail, and even Twitter, where @samuelpepys (http://twitter.com/#!/samuelpepys) now posts regular messages of 140 characters or less to his more than 10,000 followers. This approach presents historic information in small, accessible packets alongside the daily updates of friends and family.

Other historic characters on Twitter include the JFK Library's @Kennedy1961 account (http://twitter.com/Kennedy1961), which invites the public to "follow President John F. Kennedy's thousand days in office – 50 years later" and @cdarwin (http://twitter.com/#!/cdarwin), which was created by an enthusiast called David (http://www.metaburbia.com/about/). David uses a Twitter client named TheBeagle, so all of @cdarwin's messages are apparently posted from onboard The Beagle.

There are also less historically faithful characters, played for laughs, such as @henrytudor8 (http://twitter.com/henrytudor8): 'I'm the bloody king of England!'

The Old Weather project has responded to this delight in historic re-enactment by launching a new 'Voyages' feature (http://oldweather.org/voyages), which replays all of the annotations for a completed ship log on top of a map of its journey.

Figure 7: Voyage of HMS Africa

Selecting particular ships

Another important factor in Old Weather's appeal is that volunteers can choose the data they analyse. Volunteers are selecting vessels not just on the basis of what they think may be the most exciting trip but also by which needs the most attention and what contrasts with the missions they've already completed.

Post by: cyzaki on December 01, 2010
It's so exciting when you choose a new ship! Do you go for one you think might be exciting, or one with beautiful handwriting, or one that seems sad and lonely with nobody caring for it? Or do you try for the jackpot of all three at once?!

(Old Weather forum thread, ' Riveting Log Entries", http://forum.oldweather.org/index.php?topic=209.0)

Many Old Weather volunteers are motivated by an interest in naval history, or by researching their family history, and therefore look out for particular ships and time periods.

Post by: christopherhenry on October 14, 2010
I have just started work on HMS Liverpool, my father-in law sailed on her from 1916-1919, the log started in March 1915 and I am now up to June 1915, will I be able to continue right through to when my father-in law sailed on her or will I miss out if another volunteer takes over and beats me to these pages?

(Old Weather forum thread, 'General Questions', http://forum.oldweather.org/index.php?topic=9.0)

The Australian Newspapers Digitisation Program also attracted volunteers with a clear interest in local and family history: 74 per cent of its contributors are Australian and most of the contributed tags are personal names, suggesting that tags are primarily used to keep track of family research (Holley, 2009).

5. Volunteer-initiated research

All of the Citizen Science Alliance's projects start with well-defined tasks that answer a real research question. In Galaxy Zoo, 'is a galaxy spiral or elliptical?' In Solar Stormwatch, 'is that a solar storm and is it headed toward Earth?' In Old Weather, 'what measurements were taken at sea on a particular day in history?' But when you expose data to a large number of users, you also open that data up to serendipitous discovery. This was beautifully illustrated by the Galaxy Zoo "Give peas a chance" campaign. "Peas", so called because of their green blobby appearance, attracted the attention of volunteers on the forum and turned out to be a new class of astronomical object: small, compact galaxies forming stars at an incredible rate.

It's one thing to harness clicks. It's another to truly harness people power: the unique ability of humans to ask, 'What's that weird thing over there that you didn't ask me to look at?' This was more explicitly invited in Galaxy Zoo 2, with the introduction of the 'Is there anything odd?' option.

Figure 8: Galaxy Zoo 2 interface: 'Is there anything odd?'

All of the citizen science projects have a forum where members can initiate their own investigations and discuss their questions and discoveries directly with the team of professional scientists.

We introduced a whole data-analysis game in Solar Stormwatch, 'What's that?', where volunteers could look beyond solar storms to flag other objects such as comets, particle strikes, optical effects or 'something else'. The open-endedness of this game seems to have inspired the more motivated volunteers who are active on the forum. As noted by volunteer moderator Wilkinson (2010), people found "all kinds of weird stuff" in the first six months, including "camera flares, optical effects, spacecraft rolling manoeuvres and dust." The discussion tracking particle strikes drew the attention of the project scientists because the spacecraft seemed to be encountering much more dust than expected – and there is an academic paper coming soon.

Old Weather has inspired similar behaviour on the forums. Old Weather volunteers have researched one-off events of interest, such as the appearance of a comet in the sky in 1917, which was later identified as Comet Mellish.

There are even examples of volunteers identifying other data sets of interest: the number of victualling stops, encounters with other ships, and sickness records. Old Weather volunteers have become particularly interested in tracking the relationship between the 'Number on Sick List' section of the log and the well-known 'Spanish flu' outbreak in 1918.

Volunteers can also go back through their completed transcriptions to make corrections or add annotations. In working through the logs, the volunteers find that they understand the handwriting better and return to their earlier logs to improve them. Volunteers interacting on the forum have developed shared resources such as the 'Encyclopedia of Answers' (http://forum.oldweather.org/index.php?topic=1173.0), which provides technical information about abbreviations, bearings, etcetera. There are also areas where volunteers come together to troubleshoot difficult logs, such as the 'Handwriting Help' board (http://forum.oldweather.org/index.php?board=8.0).

6. Crowds and communities

While our citizen science Web sites have attracted hundreds of thousands of visits, less than five per cent of visitors participate in the forums, where much of this volunteer-initiated research happens. Holley (2009) observed an even more pronounced concentration of effort in the Australian Newspapers Digitisation Program, noting that the top ten volunteers contributed significantly more text than all other users combined.

It's clear that there is a spectrum of involvement in online volunteering projects – from small one-off contributions, like completing a ReCAPTCHA form (http://www.google.com/recaptcha) to the sustained collaboration we've seen in projects like Galaxy Zoo. To borrow Haythornthwaite's models of peer production (2009), we have both crowds – "micro-participation from many, unconnected individuals" – and communities. To facilitate lightweight contributions you need a clear, common purpose and rule-based contributions. This kind of participation is addressed by the core citizen science 'data-analysis games' and Holley's best-practice model for crowdsourcing (2010).

Table 2: Checklist for crowdsourcing, by Holley (2010)

To support a community of connected and committed volunteers finding its own purpose and producing "internally-negotiated, peer-reviewed contribution", there are three additional requirements:

1. Provide a platform for volunteers to share their data, findings and challenges
2. Ensure that professional researchers enter into discussion with the volunteers, both to set specific challenges and provide feedback, but also to respond to the questions and interests that emerge from the community itself
3. Release all of the analysed data back to the community, as soon as is practicable for the project

Post by: thursdaynext on December 06, 2010
It's good to know I'll be able to follow all the stories through eventually. I'll just have to be philosophical about it – the log has been there 95 years so I guess I can wait a bit longer. And after all, a couple of months ago I had never even heard of Old Weather and had no idea that I would ever care about what was happening on the Mantua in 1915!

(Old Weather forum thread, 'Has anyone else become Endeared…' http://forum.oldweather.org/index.php?topic=71.0)

Table 3: Extended checklist for crowds and communities of online volunteers

In our experience of Solar Stormwatch and Old Weather, we discovered that the different needs of science and history enthusiasts were less pronounced than those of the ‘crowds’ and ‘communities’ within each project. Regardless of the subject or data, a citizen science project must provide a platform for volunteers’ own research in addition to the playful and sociable interface to predefined tasks.

Acknowledgements

Thank you to our project partners from the University of Oxford, Rutherford Appleton Laboratory, Met Office and Naval-History.net – especially Chris Lintott, Arfon Smith, Robert Simpson, Stuart Lynn, Chris Davis, Philip Brohan, Rob Allan and Gordon Smith. Thanks also to Solar Stormwatch producer Natasha Waterson and volunteer moderator Julia Wilkinson. We discovered Haythornthwaite's crowds and communities model via Alexandra Eveleigh's 'User Participation in the Archives' talk at the UKAD Archives Discovery Forum, March 2011, which is available on SlideShare (http://www.slideshare.net/80gb/ukad-2011).

References

Brohan P. HMS Africa in action against Orthomyxoviridae. Consulted 1 March, 2011. Available http://blogs.zooniverse.org/oldweather/2011/01/hms-africa-in-action-against-orthomyxoviridae/

Haythornthwaite, C. (2009). Crowds and Communities: Light and Heavyweight Models of Peer Production. Consulted March 1, 2011. Available http://doi.ieeecomputersociety.org/10.1109/HICSS.2009.650

Holley, R. (2009). Many Hands Make Light Work: Public Collaborative OCR Text Correction in Australian Historic Newspapers. Consulted March 1, 2011. Available http://www.nla.gov.au/openpublish/index.php/nlasp/article/viewArticle/1406

Holley, R. (2010). Crowdsourcing: How and Why Should Libraries Do It? In D-Lib Magazine Volume 16, Number 3/4. Consulted March 1, 2011. Available http://www.dlib.org/dlib/march10/holley/03holley.html

Raddick, M. J. (2009). Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers. Last updated Wednesday, 16-Sep-2009 05:47:55 GMT. Consulted March 1, 2011. Available http://arxiv.org/abs/0909.2925

Sheppard A (7 July 2009). Peas in the Universe, Goodwill and a History of Zooite. Consulted March 1, 2011. Available http://blogs.zooniverse.org/galaxyzoo/2009/07/07/peas-in-the-universe-goodwill-and-a-history-of-zooite-collaboration-on-the-peas-project/

Wilkinson, J. (23 August 2010). The First 6 Months: Happy Half-Birthday Solar Stormwatch! Consulted March 1, 2011. Available http://blogs.zooniverse.org/solarstormwatch/2010/08/the-first-6-months-happy-half-birthday-solar-stormwatch/

Wilkinson, J. (30 August 2010). The Particles Strike Back and other strange happenings. Consulted March 1, 2011. Available http://blogs.zooniverse.org/solarstormwatch/2010/08/the-particles-strike-back-and-other-strange-happenings/

Cite as:

Romeo, F., and L. Blaser. Bringing Citizen Scientists and Historians Together. In J. Trant and D. Bearman (eds). Museums and the Web 2011: Proceedings. Toronto: Archives & Museum Informatics. Published March 31, 2011. Consulted http://conference.archimuse.com/mw2011/papers/bringing_citizen_scientists_historians_together

Editor's note: minor corrections at the request of the author: May 5, 2011.

Copyright © 2011 – Archives & Museum Informatics.