The data world has reached a crunch point. Continuing the trend of the last decade, the requirements from business of their data and by extension their data teams) are increasing exponentially. It’s not just the complexity of the requirements but the pace of change that cause challenges – months and weeks are out and sometimes even days are too slow. Simultaneously, as business, consumers and regulators become more aware, a robust and fully implemented data governance strategy is a hard requirement to ensure that the data is consistent, does not get misused, and is trusted.
To compound all of this, data engineers are between 45% and 75% less efficient than their software colleagues due to manual processes, lack of orchestration and lack of automated testing and deployments.
Many of these problems have been seen before in the software domain starting 20+ years ago and then in the cloud infrastructure domain 10+ years ago.
On a mission to address these issues Guy Adams, the Chief Technology Officer and co-founder has developed the world’s leading DataOps platform, DataOps.live.
Introduction
Guy is a self-motivated leader with over two decades of experience, guiding teams through the Agile/DevOps/CICD journey.
Contribution to the World of Data
Life was all about DevOps and then CloudOps for Guy, initially bringing his expertise of Software DevOps into the cloud infrastructure world. When he came to the data world, it brought a feeling of déjà vu. The data world was incredibly advanced in some respects however, automation, orchestrations, testing, observability, development practices, and deployment techniques were still in the dark ages! An early decision to work with the world’s leading cloud data platform, Snowflake turned out to be pivotal – its unique combination of features unlocked key DataOps capabilities and Guy has become one of a very elite group of Snowflake Data Superheroes globally.
A Career Path Destined to Challenge the Status Quo
After graduating with a degree in electrical engineering from Imperial College, London (where he was awarded the Eric Laithwaite Prize for Innovation) and time spent with the Royal Air Force, Guy found his way into a junior network management consultant role at a small company – Parallel. He worked his way up to CTO, developing the industry’s first web-based management platform and was part of the team that managed the acquisition of this company by iDirect, a leading satellite communications company. He remained at iDirect as VP of Engineering and over time added a variety of other responsibilities, including starting the first user experience department within the satellite industry.
In the early days of his career, Guy struggled with being younger than most of his peers, but this taught him how to really fight for continual progress and against established dogma, even in the face of far more experienced people. Sometimes this meant driving small but continual improvements, but other times radical and fundamental rethinks were required.
In 2018, Guy was ready for a move into a different industry and had fallen in love with the data world. He joined Datalytyx as CTO and was part of the team that lead the acquisition of this company by Mphasis. This set him on a path, with his co-founders Justin Mullen (CEO), Sam Hopper and Colin Bradford to challenge a whole industry of thirty years of legacy thinking.
Today Guy is recognized as a global authority on the principles, practices, and technologies of DataOps. He was the original architect of the DataOps.live product and led the definition of the philosophy itself as www.truedataops.org. He presents at many conferences around the world each year and lectures as part of several higher degree programs. Most recently Guy co-authored the DataOps for Dummies book.
Governance and Agility
DataOps.live has a simple mission – to automate the build, test, deployment and observation of data applications and data products on the Snowflake Data Cloud. The company does this by taking all the best principles, processes, and technological solutions of twenty years of DevOps and ten years of CloudOps and using them as a baseline. DataOps.live has then identified the challenges unique to the data world and created new solutions on top of this foundation.
This has led to numerous advantages for organizations from SMEs to the largest global enterprises.
Changing the Dynamics of Data
As businesses demand more agility, companies have reduced time to value for completely new initiatives from six months to six weeks. Releases that used to happen once every three months are now happening many, many times per day.
This has partially been driven by a huge increase in developer/engineer efficiency. By designing and building a fundamentally new data developer experience, Guy and DataOps.live have, in many cases, doubled productivity.
However, critically important for the Enterprise, this leap in agility and efficiency has not come at the cost of good governance. In fact, projects and data products managed using DataOps.live have far better governance with independent and concurrent development for any number of engineers, multi-stage peer reviews, automated testing, complete audit history for every type of change and of course the ability to roll back on the rare occasion that a production deployment doesn’t go quite as planned.
Transformation Brought by Technology
Technology is a fundamental enabler for DataOps. Until two to three years ago, the underlying technology capabilities simply weren’t there to allow DataOps to exist.
Innovations by Snowflake, such as Zero Copy Clone and Time Travel, have transformed what is possible. The team at DataOps.live is at the forefront of leveraging these to create #truedataops.
The data and application space has been divided for the last 20+ years. Databases held data and applications extracted that data and did interesting things. That model has always created performance and security problems. Developers have to bring the data to the code and if the data is sensitive or large (and large here can be Petabytes or bigger) this is very challenging. The ‘next big thing’ is moving the code to the data.
The ability to create applications using standard programming languages such as Java and Python and actually running these inside the Snowflake Cloud platform itself is game changing. Guy, his team and indeed their customers, are already starting to do this, but it means that a DataOps platform needs to support and manage not just the data but also all the requirements of the DevOps and CloudOps spaces.
Guy and his rapidly expanding team are currently focused on making the amazing DataOps.live the de facto standard in the emerging DataOps industry.
Beyond DataOps
Outside of work, Guy is an avid builder of anything impractical and over engineered from tree houses and gazebos to soapbox cars and river rafts. He is also a skiing instructor (mostly to help keep up with his two young daughters who both race slalom), an aspiring wine buff (with a love for southern Italian Reds) and enjoys the challenge of competitive crosswording!
Follow Your Dream
Asked about his experience, “Your career will never follow the path you think – looking back, my career has taken a left and right turns that I could never have predicted. If you work hard, you WILL get opportunities, the problem is that opportunities hardly ever have a big neon sign pointing to them saying – ‘I’m a great career opportunity.’ More often than not, they look like risks, sidewise moves and sometimes even steps backward. Be open to possibilities and trust your gut and the advice of the people you have built faith in.”
