From data silos to actionable insights: Innovating within legacy constraints for the digital enterprise
Market View
Sponsored by:
Produced by:
Interview with
Stephen Warner
HEAD OF BUSINESS INTELLIGENCE
ARRIVA GROUP UK TRAINS DIVISION
about
Stephen
Warner
Stephen Warner serves as Head of Business Intelligence for Arriva Group's UK Trains Division, where he oversees the development and implementation of business intelligence strategies across multiple train operating companies. With over 40 years of rail industry experience spanning operations and signaling, Stephen brings deep domain expertise to data analytics and digital transformation initiatives. He leads a team focused on leveraging AWS technologies to derive actionable insights from complex railway data systems, while working closely with IT and Digital teams to drive innovation in areas such as smart ticketing and retail strategy.
How are you approaching the challenge of extracting value from data while working within legacy system constraints?
We've adopted a pragmatic approach that focuses on getting results without overcomplicating the technical implementation. Rather than defaulting to complex API integrations, we often find simpler solutions more effective. We utilise tools like SFTP, Microsoft Power Automate, and AWS Transfer Family to automate data extraction from emails and other sources. This approach allows us to capture data at its origin and process it efficiently without extensive development work. It's about finding the low-hanging fruit that delivers immediate value while improving data quality and accuracy.
What do you see as the primary obstacles to innovation when dealing with established technology stacks?
Surprisingly, the main challenges aren't technical - they're contractual and political. We manage approximately 305 systems across our train operating nodes, and the biggest hurdles often involve supplier relationships and data ownership concerns. Some third-party suppliers see themselves as potential data stack providers and are hesitant to open up their systems, viewing it as potentially compromising their strategic objectives. We've learnt to address this by working closely with key stakeholders, particularly CIOs and CEOs of these suppliers, to alleviate their concerns. We emphasise that our goal isn't to replicate their services but to combine data for better industry insights.
Could you share an example of a successful data innovation initiative using existing systems?
During the COVID-19 pandemic, we developed a comprehensive solution for monitoring train loading in response to social distancing requirements. Without dedicated passenger counting systems, we had to be creative. We integrated data from about ten different sources, including gateline suppliers, timetables, handheld scanners, reservation systems, and CCTV. This allowed us to create an accurate model of train loading that could account for different social distancing scenarios. What started as an urgent project for Chiltern Railways has now been scaled across multiple operators, with the Department for Transport mandating similar implementations across other train operating companies.
‟
The challenges aren't technical - they're contractual and political. We manage approximately 305 systems across our train operating nodes, and the biggest hurdles often involve supplier relationships and data ownership concerns.
‟
What started as an urgent project for Chiltern Railways has now been scaled across multiple operators, with the Department for Transport mandating similar implementations across other train operating companies.
How do you balance data integration needs with the risks of changing established processes?
The real challenge isn't so much about technical integration as it is about cultural transformation. Within a typical train operating company, you have around seven functions that often operate in silos despite their interdependencies. This has historically led to a 'lowest common denominator' approach using Excel or PowerPoint for data sharing. When implementing new solutions, we have to consider these existing workflows and dependencies. There's also often a fear among staff about automation potentially threatening their jobs, and we face a significant skills gap where many 'analysts' are primarily spreadsheet operators rather than true data analysts.
How are you addressing data quality and consistency challenges across multiple legacy systems?
The rail industry benefits from having standard reference data principles for key elements like station names and IDs. This provides a foundation for joining different datasets together. We follow an agile approach to testing and validation, working closely with business SMEs to ensure accuracy. Interestingly, we've occasionally discovered that historical business modelling in Excel has been incorrect, leading to some challenging but necessary conversations about implementing corrective measures. The key is having executive support for establishing a single version of the truth and removing opportunities for manual data manipulation.
How do you envision the future of data utilisation in your organisation?
Our vision centres on establishing a single version of truth across all business functions, with standardised definitions and interpretations of data. This standardisation is crucial - even seemingly simple concepts like 'right time' for a train require precise definition. With GPS now tracking trains at 5-second intervals, we can measure delays with unprecedented accuracy, which challenges traditional definitions. A train that was historically considered 'on time' when departing one minute late now shows its exact deviation down to seconds. This increased precision necessitates standardising these definitions across all our train operating companies to ensure consistent reporting on metrics like right-time railway performance, safety incidents, customer service, revenue, and earnings.
‟
We follow an agile approach to testing and validation, working closely with business SMEs to ensure accuracy. The key is having executive support for establishing a single version of the truth.
How do you approach the visualisation and geographic aspects of railway data?
While we're a transport operator, we've found that geographic visualisation isn't always the most crucial aspect. We use GPS data primarily for specific operational purposes, such as analysing unexpected train deceleration. We've developed models that compare actual train speeds against expected line speeds, allowing operational teams to investigate anomalies. By combining line graphs with maps, they can click on deviation points to see exactly where a train slowed down and investigate potential causes like signal changes or path regulation.
We're also exploring opportunities to enhance our understanding of passenger journeys through geographic data. We're working with partners to analyse mobile network data, which provides insights into first and last mile transportation choices - how passengers get to stations and their onward journeys. This helps us understand catchment areas and travel patterns, though we ensure all data is properly anonymised.
How are you managing the upskilling of your workforce to better utilise data systems?
We've established a BI community and maintain a team site for sharing knowledge and innovations. We conduct quarterly workshops on new technologies and the latest platform innovations, while also reviewing job descriptions and role profiles to ensure they align with our evolving needs. Our training approach is always problem-focused - we've found that generic training without immediate practical application doesn't deliver value. Instead, we tie training to specific projects or innovations, ensuring people can immediately apply what they've learned.
We maintain a comprehensive register of training records and conduct monthly communications to keep skills current. We've also developed our own internal test data sets, as we've found that using industry-relevant data rather than generic examples helps people better understand how to apply tools to their specific challenges. This approach helps ensure that our team can effectively work with our AWS stack, including S3 for storage and QuickSight for frontend solutions, while maintaining the specific knowledge needed for rail industry applications.
about
Arriva
Group
Arriva is one of Europe's largest mobility providers and a subsidiary of Deutsche Bahn, operating various transportation services including trains, buses, and coaches. In the UK, Arriva UK Trains division manages several train operating companies, focusing on delivering efficient, sustainable public transport solutions. The company combines traditional railway expertise with modern digital capabilities to enhance passenger experience and operational performance through data-driven decision making.
We are the global market leader in geographic information system (GIS) software, location intelligence, and mapping.
Esri UK is the exclusive provider of Esri solutions in the UK. We take a geographic approach to problem-solving, brought to life by modern GIS technology. We are committed to using science and technology to build a sustainable world.
Share this page
