Easy to find and follow: The Times and The Sunday Times Twitter directory
Lots of our journalists at The Times and The Sunday Times use Twitter to break news and engage with readers about what they write. Some of our journalists have built up large followings whilst others have been contacted with stories as a result of the loyal and engaged following they have.
Not only that but we’ve found that their use of social media has caused people to become subscribers, especially around big events such as the death of Margaret Thatcher. At numerous points during the last year, the conversion rate of visitors from social media (surprisingly if we’re honest) was stronger than other channels like search and PPC.
Anecdotally too, social is increasingly a way in which new subscribers begin their relationship with The Times and The Sunday Times. We often speak to new readers at Times+ events who say they were persuaded to subscribe having followed a journalist or a section account and seeing content that they’d like to read but can’t. Twitter, it turns out, is a great shop window for our journalism.
With that in mind, we figured it would be beneficial to try and increase the number of followers across our journalists and section accounts. To do so, we decided to create a Twitter directory for each title to make it easy for readers to find and follow the accounts that interest them.
The directory is pulling from two Twitter lists, one for The Times and one for The Sunday Times and has been built using a Django backend plugged into Twitter API to allow the social media and community team to add and remove journalists and edit their titles without using a developer’s time.
We’ve promoted it every day in our tablet editions and on the homepage of both websites (although it’s a bit hidden at the moment). Unique visitors have been a steady on The Times version since launching in September and, whilst the number of followers driven from the directory is difficult to measure accurately, average dwell time is over 1 minute which we think is pretty healthy.
We’ll be keeping an eye on The Sunday Times version which launched this week and will make tweaks to the design and UX accordingly. We hope to make it easier to find individual journalists and sections by adding anchor links and a search functionality. Also we think there’s value in linking the Twitter directory to the news agenda by explaining why you should follow certain journalist on that day or week. We’ll post about any changes here on the Times and Sunday Times Digital Development blog.
Thanks for reading
Ben (benwhitelaw), Henrique and others from the Digital team
Crossing the Gender Divide - A Sunday Times Data Team exclusive
When a group of data journalists are asked to look at a large and complicated pile of figures, we go straight to our tool box. Just as a plumber, presented with a leaky faucet, shakes his head, calculates how much the client can afford and goes to fetch his tools. But what happens when we are not presented with data? What happens when we are given an idea?
A lost generation
The Sunday Times journalist, Kevin Dowling came to our team with an idea. An American Report called ‘Failure to Launch: Structural Shift and the New Lost Generation’ was published by three academics. This outlined how, in the US, it is taking people six years longer now than in 1980 to reach the median income - an accepted marker for financial independence. With this lost generation in mind, Kevin wanted to see what findings might emerge from UK population data. We accepted the task and took on an additional challenge, to work in news time not academic time.
Luckily, our data journalism team has an Italian-bred, polo neck-wearing statistician in the form of Stefano Ceccon. He collaborated with fellow data journalists Megan Lucero and Nicola Hughes to gain press access to the the UK Data Service and search for available data on UK population income over the years.
While there are several datasets that capture income data, we found the Living Costs and Food Survey, to be the richest dataset for demographics and historical income data.
The Living Costs and Food Survey is a household expenditure survey which has been conducted, in different forms, each year in the UK since 1957. The survey is collected by the Office for National Statistics and produces the biggest pool of UK social data after the Census. For example, in 2011, a total of 11,484 households were selected, of which 5,293 households responded to the survey.
This dataset had two key variables from which we could build a new extended dataset that linked up the data for each year. One was the case reference number variable - indicating the person interviewed - and the other was the household reference number. We gathered, cleaned and plugged around half a million rows of this data into the statistical software package R. We were thus able to isolate Sex, Age, and Full-Time or Part-Time Employment status for analysis.
We converted the income variables into annual pounds earned and filtered out people not working Full-Time. Once we looked at the distribution of income we soon realised that as expected, this left a skewed distribution with several outliers - there were a few atypical people who sat at the top of the distribution with very large incomes and on the other end, those with very little. This is what led to our decision to adopt the median income as a reference value for the population.
A bit of bootstrappin’
We were now presented with the issue of having to estimate the confidence intervals for the median income and soon found our saviour in bootstrapping. This is a well known statistical resampling technique that allows the estimation of figures and their confidence intervals. In the calculation of the medians, we also included the weights (where available) provided by the ONS, to compensate for non-response and the population distribution in terms of region, age group and sex.
Nicola expressed our relief best when she said: “Ah, a little bit of bootstrapping will cure whatever ails ya.”
This solution had opened the door. This technique allowed us to calculate with confidence the median income, the reference point we needed to establish financial independence for the population in a given year and allow us to analyse income over years, gender and age.
We had one final step to take to make sense of the median income plots for each age and clearly see a trajectory for the year. We used a polynomial regression technique to fit a smooth curve so we could see the age at which the median was reached for each gender and year.
The gender gap
There were several key findings that emerged from the analysis, a major one being the closing income gap between men and women. Women only reached the median income of the population in 1998 and since then have steadily been getting a larger piece of the cake. Another interesting finding was the shift of the age at which the population reached the median income. From 1971 to 2011 the age at which the population was reaching financial independence (median income), had shifted almost four and a half years.
These findings led to the publication of The Sunday Times piece: “It’s taking too long to grow up” and the Digital exclusive piece: “The Gender Gap”. The latter piece was accompanied by an interactive made by our team, in conjunction with a developer from The Times and Sunday Times Digital Team, Callum Christie.
Feel free to contact us with questions or comments.
-The Sunday Times Data Team: Stefano Ceccon, Nicola Hughes and Megan Lucero
Product testing @The Times Crossword Championships
Every year, the crossword elite (or ‘solvers’ as they are known within the community) congregate at News UK for The Times Crossword Championships. A truly underground event, it pits the sharpest brains against each other in a knockout tournament that culminates in the Grand Final and the Annual Champion being crowned.
The Grand Final - no other competitive event resembles a school exam quite like The Times Crossword Championships
As Robert Hands observed in his presentation of the winner’s trophy, nothing inspires the same levels of engagement and passion amongst readers of The Times and The Sunday Times as puzzles. Errors or bugs in the gameplay on our digital products (particularly the tablet) rank as some of the most frequent complaints from customers. When a decision was taken to make access to The Crossword Club available only as part of an annual subscription, the level of outrage almost necessitated police protection for the manager of Customer Service.
Transferring the full range of puzzles into the digital space, whilst at the same time leveraging the new opportunities the medium affords has been a challenge for us. The piecemeal development of apps and websites has lead to a fragmented, disjointed puzzle experience. Ironically, the content that consistently delivers the strongest response from our readers has received the least amount of attention and co-ordinated development.
This is partly because the majority of the Editorial and Digital Product teams are not themselves solvers and partly because a truly long term solution to online puzzles requires the development of a complete platform and not just a new product feature; a challenge for a business that works at the speed of the news cycle and for whom long term objectives are measured often in days rather than the months that are required to build a software service of this nature.
David from Digital Marmalade demonstrates the new web interface to an earnest-looking solver
However, thanks to some much need moral and financial support from our red-topped cousins at News UK (who have similar requirements in this regard) we engaged Digital Marmalade, a local agency, to finally help us achieve our vision. The vision was to do for online puzzles what Amazon’s Kindle has done for online reading: deliver a channel agnostic solution that allows solvers to access puzzles anywhere they choose (be it on smartphone, tablet or web), with consistent gameplay and seamless content recognition. In short, to allow someone to start a crossword on their phone in the morning, continue on tablet over lunch and finish it at home on the web.
After several months of intensive requirements capture (from ‘solvers’ and ‘setters’: the puzzles editors), detailed analysis of the nuances of online gameplay and huge support from the legend that is Peter Biddlecombe, the ST Crossword Editor, we are finally ready to launch v1.0 of our vision. And where better to test drive than at The Times Crossword Championships?
A truly multi-channel solution
Built as ‘liquid’ HTML, the new solution is injected as a ‘webview’ within apps and is optimised for browsers, 7” tablets (eg Nexus, Kindle Fire and iPad Mini), 9” tablets (original iPads) and smartphones. Gameplay is a construct of keyboard (for data entry), the grid (where words are displayed and completed) and clues. Each form factor requires a different configuration and poses different challenges due to constraints of space.
A current iPad solver scrutinises the design of the new crossword solution
Nothing beats feedback from real users and there are few more forthright than The Times crossword solvers. It was on the design choices (and tradeoffs) we had made that we were most keen for feedback on.To facilitate testing, Peter Biddlecombe provided 2 crosswords from yesteryear and Digital Marmalade input them into the platform.
Having gathered every spare laptop, tablet and smartphone we could find we positioned ourselves conspicuously next to the competition. Many attendees we approached were diehard print readers and had no desire to migrate to online puzzles. However, a significant proportion were habitually doing crosswords on their tablets (and to a lesser extent the web) and were keen to examine the new solution. Understanding how best to move between clues (some prefer arrows to ‘tab’ through each one sequentially), why it is preferable to have the entire grid visible at all times and why it’s important to be able to ‘pencil in’ possible answers before committing to them are insights that can only come from the habitual solver.
David from Digital Marmalade demonstrates his new solution in 7” format on Kindle Fire
The feedback and observations we received on the day provided vital insights into the benefits and limitations of the new solution and will be incorporated into the ongoing development cycle of the service as we approach its eventual incorporation into The Times and The Sunday Times digital products. Additional testing for the tablet in particular will be carried out within a wider subset of existing solvers to ensure that the eventual transition is a success. So specific and nuanced are the finer details of the gameplay on this product that any sudden changes, if ill considered, would be disastrous in terms of customer response.
Crosswords on the smartphone - the missing link of the multi-channel experience
The new frontier for the solution is undoubtedly the smartphone. None of the smartphone apps within The Times and The Sunday Times bundle feature puzzles so there is no context or frame of reference for customers to evaluate their likelihood of adoption. Initial research indicated a surprising number of customers interested in a smartphone solution however, amongst those we spoke to at The Crossword Championships, few seemed able to comprehend the possibility of solving crosswords on their smartphone. As it would be a value-add to the current product, it is likely that we will surface the new solution in this channel first.
Paul Jackson - Digital Product Manager
Pat Long - Head of News Development
Building the Westminster Watchdog at #NewsHack
A fortnight ago a team from the Times attended #newshack, a two day event organised by the BBC that pitted news organisations, universities and more against each other to see who could come up with the best digital project. Nicola Hughes, data journalist at the Times, has written a wider summary of the day as well as more general thoughts on hack days.
We were awarded ‘Best Journalism Tool’ for our project, creating the Westminster Watchdog, which combined natural language processing, machine learning and gamification to scrutinise MPs’ Parliamentary conduct.
Simply put, we scraped information from MPs’ register of interests, and created a tool that allowed both an algorithm and user input to flag if there was anything suspicious. The game aspect came from training up a dog, with prizes awarded to it depending on how much help the user provides.
The system we built can be applied to many things - we could easily add to it by adding MPs’ twitter feeds, public Facebook posts etc. It was very rough and ready but after two days hard work by everyone on the team it was enough to impress the judges. We hope to refine and improve the tool in the future.
Hack days can be frustrating, and restrictive - but ultimately they can teach participants so much as you’re forced to think on the fly and produce something that makes sense.
Thanks to BBC News Labs for putting on a great event (though we’d like more plug sockets next time!) and congratulations to our team Aendrew Rininsland, Nicola Hughes, Mario Cameira and Matt Taylor.
Working with longform for ‘My year with Malala’
For the past two weeks we’ve been working on presenting a digital version of Sunday Times foreign correspondent Christina Lamb’s longform piece about her relationship with Malala Yousafzai. You can view it here.
The piece itself is fantastic - highly personal, with lots of great anecdotes that lend themselves to be augmented with pictures or video.
We gathered all the assets we had and began wireframing some designs which sought to make best use of multimedia.
It was built using a set of custom WordPress-based tools that were designed in-house following some earlier experiments with longform content.
All the content was edited and managed in the popular WordPress CMS, with visual layout tool SiteOrigin Page Builder used to arrange the piece’s sundry elements around the page. The finished piece was then exported to the production server as static HTML for performance and security reasons.
Beyond that, the Twitter Bootstrap 3 framework was used for a few components (namely the navigation bar and modal windows), with the project’s many videos streamed from Ooyala.
One of the main things we’re keen to do is begin to build a framework that allows us to simply present longform pieces without having to create bespoke designs every time. We think that there’s a lot of value in being able to produce something simple that works that we can build on in the future.
- @josephstash - @aendrew
Livefyre comes to the UK: The Times @ official launch of the Livefyre London office
In September, Livefyre officially inaugurated its London office with an event at the Soho Hotel and a Q&A on ‘Emerging Trends in Social Media’ moderated by Forrester’s Anthony Mullen and featuring The Times and Sunday Times’ own Nick Petrie in conversation with Livefyre’s CEO Jordan Kretchmer and Time Out’s Dave Cook.
Nick Petrie at the Livefyre UK launch @ the Soho Hotel.
The panel discussed how they quantify the value of social engagements in terms of the bottom line, how to leverage social media to increase the reach of your journalism and the complexity of integrating social engagement strategies with a paid-journalism model.
Nick discussed the use of Livefyre’s technology to support The Times’ recent Ashes Social Hub and his plans for future hubs to support upcoming media events.
Nick debates the future of social media with Jordan Kretchmer, founder of Livefyre.
Re-imagining comments: The Times wins hack day with comment thermometer
A team from The Times won a two-day journalism hackathon, in which six teams from UK newsrooms (The Guardian, The Daily Post (Trinity Mirror), the BBC, Haymarket Business Group and Metro), competed to build digital content around the theme “Re-imagining online commenting: an alternative to a thread below the feed”.
The two-day hackathon, which was held at the Guardian, was organised by the Global Editors Network, and supported by Google.
The hackathon is part of GEN’s Editors Lab global programme, and are designed to bring individuals in the newsroom together to inspire innovation.
The Times team, consisting of Henrique Alves, Ben Whitelaw and myself, designed and developed a basic prototype for The Times ‘Comment Thermometer’, which rates comments using sentiment analysis.
It sought to solve a problem that readers often face which is that it is hard and time-consuming to read lots of comments under an article.
Most comments on The Times are currently posted on the site early in the day and whilst readers can filter comment threads, the options are crude; by time (newest/oldest), most ‘liked’ or recommended comments. The most recommended is useful but is often not representative or meaningful - it might be a quip or gag that has been written to attract ‘likes’.
Our challenge was to create a way of ‘taking the temperature of a comments thread’ and to open up comments to all readers - journalists, commenters and non commenters, making comments accessible for everyone.
The Times ‘Comment Thermometer’ would better gauge reader reaction and make it easier for readers to filter comments based on their underlying sentiment, using a sliding scale.
The thermometer assesses the sentiment of each comment by comparing the language in the comments with a dictionary of positive and negative terms, qualifiers and inverters.
A dynamic ‘sentiment indicator’ panel, appearing at the top of the article, aggregates ‘the temperature’ of readers sentiment, and all comments posted. The panel background colour will vary depending on the current consensus of opinion at time of reading.
We hope that by allowing Times readers to filter comments in this way, we widen the net of people who interact with comments beyond those who write them.
The Times team will be invited to the Editors Lab Hackdays Finale at the GEN Barcelona Summit 2014, to compete against the winners of the twenty national Hackdays being hosted all over the world since August 2013.
You can watch the pitches of all teams here (Times team is up first and coincides with a protest in the Guardian offices!)
Building the Conference Companion for political coverage
The Times Conference Companion is a lightweight mobile website allowing subscribers (and non - subscribers that light register) a fast and easy way to keep up with conference news and gossip.
We wanted to find a way of bringing a Twitter-like stream to readers that was heavily curated and included the best of our political coverage and analysis. The Conference Companion combines opinion and news from our website with our journalists’ tweets (and a few other key figures).
Non-subscribers can register for free and get limited access to the articles placed in the companion’s stream.
Under the hood, Conference Companion is two parts: a Django backend running a REST API, and a HTML frontend that ferries requests to and from the backend.
This was the first time we’d really used the Python-based Django framework in a project, and were pleasantly surprised by its flexibility. Django REST Framework was particularly great — using it, one needs only define endpoints and how they output data in order to get a web-accessible way of communicating between disparate systems.
This time round - simplicity was our focus. We worked on pulling in articles from our website, and creating a nice mobile design, but we already have plans for more functionality to be built into a future iteration of the site.
This project was coded by Aendrew Rininsland & Henrique Alves and managed by Nick Petrie and Joseph Stashko
Preparing for the future
On June 22 1974, The Times moved from its building in Printing House Square to new premises on Gray’s Inn Road. A short piece in that morning’s paper described the old offices as they were in the late Victorian era:
“Coal fires roared in open hearths. A thick layer of soot covered everything. Photographs show the editorial conferences of patriarchal figures with pipes and beards enthroned in leather arm-chairs in a book-lined room like a superior club or the upper room of some top person’s ivory tower. It is no legend that the wine cellars at Printing House Square were excellent…”
Above: the current Times conference room in Wapping
The Times offices hadn’t changed much by the 1920s, as Graham Greene described in his autobiography A Sort Of Life. Recalling his brief tenure as a Times sub-editor, Greene wrote of “the slow burning fire in the sub-editors’ room, the gentle thud of coals as they dropped one by one in the old black grate…”
Just as the days of coal fires, wine cellars and leather arm-chairs now seem impossibly remote, so we have seen huge technological, business and cultural changes in the short period since The Times last moved offices three years ago.
At the end of 2014 The Times will move once again, from our current base in Wapping to the building nicknamed the Baby Shard, a short distance over the Thames.
Above: ‘The Baby Shard’
Along with a small group of colleagues from the Times Digital Development team, I’m currently looking at how we can use the move to London Bridge as an opportunity to build a newsroom of the 21st century, equipped for our journalists to tell stories and deliver news in the best possible way, while allowing our readers the opportunity to read our editions wherever they find most convenient.
This piece of work will form part of our Newsroom 360 initiative, but is party theoretical – the new building isn’t yet finished and the move will finally be completed early in 2015. But in preparation we’ll be reviewing models from other newsrooms around the world and talking to colleagues from other newspapers in our parent company to see how they are preparing themselves for further change in the way that they use digital technologies to help their reporting.
Over the coming weeks I’ll be sharing some of our findings on this blog wherever appropriate. The future of the world’s greatest news organisation has never been in doubt. With this move, it’s also never seemed more exciting.
Pat Long, Head of News Development, The Times and Sunday Times