Getting to where we are now

Susan Lambert's picture

This is a joint blog written by all three partners

10 Most Wanted is a first generation Digital R&D Fund for the Arts project which combines social media and online gaming to motivate the public to search for unrecorded information about cultural artefacts. It outsources tasks commonly undertaken by professional curators, thereby taking crowdsourcing to a new level: while most crowdsourcing projects ask the public to complete simple tasks that do not require sustained engagement, 10 Most Wanted asks people actively to undertake research.

We knew we were setting out to do something that had not been done before and that it would not be straightforward. We were pleased when NESTA explained at the first Programme Networking Event that it was understood that such projects at the leading edge of experience would not always go to plan and that what was essential was that we document the decisions that led to any deviation from our initial proposal.

The project began in May 2013 and was meant to last 12 months.

The original intention had been to launch the game’s website during October, leaving three to four months for people to play the game and two to three months for the project team to review the findings and write the final report. Instead, we approached the development of the website in a different way (decision point 7).

Rather than hiding the development website behind a teaser site and later doing a ‘Big Reveal’, we decided instead to make the site in a publically accessible way from the outset. This would enable us to link social media accounts to the website straight away in the knowledge that continuity would be maintained and that the audience could play a part in shaping the site’s development, thereby maximising their sense of ownership and emotional engagement with it. Additionally it ensured continuity in gathering statistical user data.

As we got closer to the planned public launch date in late October we stepped-up the testing, in particular, the testing of social media channels and APIs.

This activity attracted the attention of people through social media and they started to follow what we were up to, discovered the website and immediately dived in and started to post their discoveries. These early adopters provided us with lots of useful feedback and led us to make improvements to the navigation, to how some of the systems worked, to how the status of artefacts was handled, to the user ranking tables and various other things that were tweaked and polished. They also made us aware of the importance to address copyright issues in a watertight way, as one key contributor, who joined the project when not all our social channels were fully set up, left the project after feeling misled about the way his contributions were reproduced on the project website.

The early influx of traffic delayed the final completion of the website. Given that it was already publicly accessible (decision point 37) and attracting an audience, we decided to turn this to our advantage and arranged for the formal launch of the website to take place a couple of weeks later than planned: on 15 November 2013, at the Museums Computer Group’s UK Museums on the Web conference at Tate Modern, attended by 160 people and with the appropriate theme of ‘Power to the People’.

Informed by feedback from our survey group (Tools and Methods 2) and collaborators, we carried on making changes and adding features to the website ever since the launch, which is consistent with our iterative development approach.

However, the focus of our efforts shifted with the beginning of the new year to getting as many players as possible to engage with the website and our social media channels. Based on findings from our literature review we have tried various engagement strategies so far:

  • We wrote press releases, which were picked up by a number of smaller publications, websites and the plastics industry press.
  • We contacted a range of radio and TV programmes with background information about the project and Wanted posters for topical objects relating to the Winter Olympics and the upcoming Football Worldcup (albeit without success so far).
  • We posted information about the project in various social media outlets, forums and mailing lists.
  • We contacted individuals at various museums and archives, as well as special interest groups such as Local History Societies, the University of the Third Age, and the Plastics Historical Society.
  • We produced leaflets, posters and badges and presented the project at various local and national events.

In total, we estimate to have reached a minimum of 10,000 people with information about the project.

 

The argument for extending the project

It is now mid-March: we have solved five cases and discovered a total of 42 facts about the featured objects, but public engagement with the project has been disappointing, with just a handful of stalwart players and the odd stellar contributor.

While the overall pattern of only about 5% of registered users actively contributing is consistent with the experiences in other crowdsourcing projects, 10 Most Wanted can only reach its full potential if we succeed in engaging more people to participate.

At this stage we are not clear about the reasons for the low engagement. A comprehensive evaluation of the project website, terms and conditions, consent mechanism, contribution mechanism and overall player experience is underway and could be completed by the agreed end of the project.

However, there are additional engagement strategies we have identified (see below) but not tried out so far. Furthermore, we're still working on getting broadcast media interested in the project, which in other crowdsourcing projects has been identified as a pivot point in participant engagement: both Transcribe Bentham (Causer and Wallace, 2012) and  Galaxy Zoo (Raddick et al., 2010) describe how the number of participants surged after broadcast media had reported on the projects.

Clearly a project involving participants in complex tasks relating to a specialist resource does need time to build an audience. One of our most valuable contributors made an interesting point when invited to blog about his experience on 10 Most Wanted: ‘Twelve years ago I needed to know something about a 1920s product. I wrote a one-page web site that contained a question and my phone number. Three years passed, and then one day I got a phone-call. An elderly lady said, I read your web site. I know the answer to your question.’

While we could finish our evaluation and write a final report in time, we believe that an extension of three months will make the research findings of our project more valid and provide a much clearer picture of the actual potential of our methodology.

 

New engagement strategies

In addition to redoubling our current engagement efforts we will test two new strategies:

Central to our methodology, like the FBI and its most wanted criminals, has been focusing on a limited number of objects and it has been our policy to leave them up even if players are not engaging with them. While continuing to stick to this basic principle, we will swap out objects more quickly.

Criteria for swapping objects could be:

  • the amount of attention they get (e.g. swapping out objects that are not eliciting comments for a whole week)
  • their relevance to current events (e.g. swapping in objects that can be related to themes trending on Twitter). We will test this approach for several weeks and try to match any spikes in engagement (measured via social and web metrics) to object swaps.

The second approach we will test is involving schools as a new audience, which has proved to be successful in the Transcribe Bentham crowdsourcing project (Causer and Wallace, 2012). Key to this strategy is to make explicit links between player activities and the curriculum.

Engagement with 10 Most Wanted can provide a range of learning outcomes including:

  • Increased ability to look closely at and interrogate objects.
  • Greater appreciation of how things are made – they are designed and manufactured, they do not just happen.
  • Improved research skills.
  • Enriched engagement with the internet.
  • Application of social networks to achieve something useful.

We have developed draft teachers’ notes and a student’s brief and the next stage is to have these reviewed by teachers of different age groups. Initial discussion has suggested that they could be useful for children from ten years and upwards although a range of briefs targeted at particular age groups and subject areas will be required.  It would certainly be a win-win situation were the extension of the project to lead to it having a role as a learning tool.

 

Summary

In summary, we decided to extend the project by 3 months (subject to agreement by the Fund) in order to give us more time to grow a participant community and to test new engagement strategies. We believe that the extended runtime will help to better develop the potential of our approach and lead to more realistic findings.