Hi, I’m Nick Ryan, Head of Test and Release in the IT Development, Test and Operations area of HMRC. We have one of the largest IT estates in Europe and I’m always happy to share some of our lessons learnt in keeping 600+ systems up to date. This is the first time I’ve blogged and I’m sharing some tips for successful tech testing.
Working in HMRC digital
When you work in HMRC you soon realise we have regular events that are hugely important to our customers, whether it’s the start of the new tax year in April, the Self Assessment peak on 31 January (9.59 million returns in total) or the annual renewal of tax credits on 31 July (available to 4.4 million families). My role is to lead on the testing and release of IT updates for those key business events, to make sure they run smoothly without any adverse impact on our customers.
HMRC are part way through our transformation journey to create an IT architecture based on a small number of resilient and flexible platforms. This means we can sometimes test and release constantly in the background without this work causing any downtime and disruption to our millions of customers and sometimes we have to be a little more creative with the testing and deployment for our older systems to minimise downtime and disruption.
Big events like the new tax year pose the greatest challenge with changes and updates coming in from many different areas within HMRC. We test them all separately, sharing data and rules from shared services before bringing them together on the one system. And as you might have guessed making numerous changes to various systems using different technology stacks and hosting solutions involves months of preparation and testing.
We’ve had a bit of practice at this now so here are my top 5 tips for a successful mass test and deployment of the type that happens in HMRC every month of the year.
Keep your customer key
All IT updates should be designed by people who understand your customer needs and priorities.
In our case we work in partnership with our customer-facing teams. These are the people who interact with our end customers everyday and have a real insight into what works and what doesn’t. But they’re also our customers too, we have to make sure the IT we provide is right for them and their role within HMRC.
Collaborative working gives us an understanding of the rhythm of the department’s life and an ability to make choices on what to test and deploy and when. It’s invaluable in helping us maximise the usefulness of the updates whilst ensuring we minimise the blast radius of a failed change.
And of course relationships aren’t just for a particular event they are for life so every colleague we work with on this with becomes a colleague we can work with again next time.
Get great governance
Have a plan, a structure and stick to it.
For example we run weekly test and change and release forums with all the technology areas of HMRC. It’s an opportunity for the community to come together, find out what other teams are working on, share stories, and meet each other face to face. Plus do the hard yards together of formal review of changes, identifying the links and dependencies and what needs to be deployed together and what can just go in.
Sometimes you need more flexibility too, so we support our permanent governance with transient groups that form quickly to sort specific issues and dissolve just as quickly when their work is done.
As we move towards the final weeks of a delivery we have structured reviews with technical and business authorities based on risks, issues and dependencies to ensure the deployment train is on the right track.
Cherish your rituals
We want to constantly learn, innovate and improve and we understand great delivery means being able to do things over and over again. So we’re fanatical about maintaining and improving our corporate knowledge, working practices and lessons learnt, and know what works for us in our environment and our culture. We then use those strong foundations, based on proven rituals, as a core mitigation of our risks for key business events.
Make monitoring mega
Don’t even attempt something unless you have a way to monitor the impact of what you’re doing. We have world-class monitoring systems in HMRC to see what’s happening second by second on our platforms and servers. It gives us 24/7 visibility to ensure that if there is an issue we spot if fast and fix it quick.
Just as important as the tech is a well-tuned event and incident management process that allows IT professionals space and time to think and analyse the event or incident and then work together on the fix.
I get out of bed in the morning and go to work in HMRC because the chance to be part of its purpose and contribute to us all being part of a civilised society is hugely important to me. As part of that I want all the great people in HMRC working towards a successful delivery to enjoy it. There’s no point in having world class tech with unhappy colleagues. So my last top tip is be strong on your culture and values.
In HMRC we focus on areas like professionalism, respect and innovation and spend time every day working to facilitate and enable that culture. Your culture and values will be unique to your organisation but whatever they are they’ll be a key part in engaging your colleagues and satisfying your customers. Guard against forgetting about them in the rush to deliver.
I hope that all of the work we did on the recent new tax year gave you, as a taxpayer and therefore one of my customers, a good experience. If any of my thoughts in this blog ring any bells with you then I’d love to hear from you.