IT sprawl is a common issue facing many large government entities. What were the particular challenges in Ohio?
Stu Davis: Across Ohio State government, we have 9,000 servers, more than 2,000 applications, and a lot of storage sprinkled all over the place. There are twenty-six different cabinet agencies, and they each have a CIO that does not report to me. There were thirty-two data centers in use even though we do have a primary data center. When I came in, I found each department doing its own thing, but wrestling with the same budgetary issues. Through IT optimization, we are changing the way the state does business and lowering the overall cost of IT.
Government agencies are more often than not in competition for resources. How were you able to get everyone on board and communicating effectively?
We had great support from the administration. When you have the department of administrative services, the office of budget management, the governor’s office, and the board of regents all in alignment, you can really make a big impact. To meet the needs of each agency, we created a technology board by segmenting those twenty-six different CIOs into five different lines of business. Each line of business has a lead on the technology board that carries information between the agencies and us. We had talked about disaster recovery for over six years, and we could never get the agencies to give us all the information we needed. By going through those lines of business and the technology board, we pulled that information together in three months. Agency participation has been exceptional and crucial to the success of our IT optimization initiatives.
Five Reasons Tech Talent Should Consider the Public Sector
1. Advancement Opportunities
In Ohio, 32% of the IT workforce is eligible for retirement. Budding IT professionals cannot only get in on the ground floor, but have the chance climb the ladder as well.
2. Application Development
Government IT departments are shifting their focus outward, giving teams the chance to build and develop applications that impact citizens.
The workforce at large is pivoting away from a traditional thirty-year career at a single company. IT work in the public sector offers a chance to start a career or give it a refresh—it can also lead to a diversified résumé.
Projects in the public sector have the possibility to positively impacts tens or hundreds of thousands of people. In the case of Ohio, IT projects can touch more than 12 million citizens.
5. Positive Impact
Like any public position, working in government IT is a public service, and working in the public sector is a way to give back to the community
Where do you start with a project this size?
The first step was e-mail. There were nineteen different e-mail systems in use, and the inability to communicate efficiently from agency to agency was out of hand. Next, we wanted to migrate everything onto our centrally managed mainframe so that we could more easily do some much-needed upgrades. Server consolidation was also important. We needed to reduce complexity in order to have a single, statewide posture for security and disaster recovery. And to accomplish all of this, we needed to update our primary data center, the State of Ohio Computing Center (SOCC)—a 210,000-square-foot data center that was designed back in the late eighties. In February 2013, we began updating the SOCC in preparation for the migration of servers.
Where do you stand on those improvements today?
We have consolidated all e-mail systems, with about 64,500 e-mail accounts migrated to the center. The necessary building and facility fixes for the SOCC were completed in March 2014. We now have a large space on the second floor, about 70,000 square feet, where we believe we can host every IT asset of the state agencies. Since then, we’ve migrated about 2,300 virtual-server images and about 1,500 servers into the SOCC.
Each migration is a learning process, and we’ve had great involvement from a number of agencies. Each time we go through this, we take a step back, look at the lessons learned, and apply them to the next. We also repurposed the third floor to provide collocation services for organizations such as Ohio State University, so they can leverage Tier 3 data-center services, and others are using it for a disaster-recovery site.
Square footage of the State of Ohio Computing Center (SOCC)
DNR servers migrated to the SOCC over one weekend
Reduction in IT infrastructure spending between 2013 and 2014
I would imagine all of this technology consolidation also required some personnel shifts.
Definitely. Even though the department of administrative services is the central IT provider, it was not staffed appropriately for all of these new initiatives. We posted positions and hired the additional staff from other IT agencies in state, and 86 percent of the central IT workforce has experience from another agency. That interagency team quickly changed the culture, removing some of the bureaucratic hurdles that can plague large projects.
Technology changes so rapidly year over year. How are you planning ahead now to avoid the need for yet another overhaul in ten to twenty years?
As we centralize, we’re building out our private cloud. And as we reduce the complexity and embrace standardization, we are making the operating system and the configurations the same across the board. That increases our flexibility and agility looking forward to new cloud services outside of what we currently provide.
What was the cost of this project to Ohio taxpayers?
There were no cost increases to the IT budgets of the various agencies. We were actually dealing with a 10 percent cut across the board, so most of the things we’ve done have been by leveraging existing resources. When we bring in virtual images, we repurpose the servers they came in on, do the virtualization, and then put those back into the server farm. There wasn’t a big budget for IT optimization. It was a collective pooling of resources.
More efficient procurement and planning also helped reduce our costs. Beginning in 2012, we put policies and procedures in place to ensure that agencies were making the appropriate purchases. That came to fruition between 2013 and 2014, when we saw a drop of $36 million in our IT infrastructure spending. Some particular agencies were getting ready to make large storage purchases. We got ahead of those costs, migrating their storage to the SOCC. That saved $400,000 in just one instance.
Given the improvements made so far, what is on the agenda for the next couple of years?
Our priorities for fiscal year 2015 are the continuation of the migration of our server infrastructure to the SOCC and building out our private cloud. We will also see a flip in how we spend our money.
Before, spending was focused on infrastructure and operations, with a very small amount on our public-facing and business-facing applications. We want to flip that, so that we are spending 40 percent on infrastructure and investing 60 percent on what matters.
We’re starting to see that trend; 35 percent of spending right now is focused on applications that bring value to the citizens and businesses of Ohio. One example is integrated eligibility, where citizens can become eligible for Medicaid online. Citizens are more educated, informed, and likely to be taking advantage of the technology at their fingertips, so rapidly deploying these applications is crucially important.
When will this consolidation be completed?
It will never be quite done. There is always going to be continuous improvement. We told the governor in 2012 that this particular wave would be five to seven years. We are two years in and making significant progress.