Database Transformation Playbook
This playbook is designed to help you transform your database technology and overcome any obstacles on the way.
- Each play is an integral part of database transformation decisions.
- All these plays in total are essential to the success of these decisions.
- Note: GSA will continue to publish updates to the playbook with best practices and lessons learned.
- GSA will publish an updated case study article/playbook detailing some of the challanges and lesssons learned from the TMF Database Transformation Project. We expect to publish the case study NLT September 2019. Thanks and stay tunned!
- Last Updated on 8/6/2019
1. Know your inventory
Know what databases and data you have. Know the producers and consumers of the data
Weigh options against business priorities to see which projects will make the biggest impact
3. Set goals
Set achievable, focused goals that move you towards your priorities
4. Pilot solutions
Start with smaller, engagements and you will learn from those as you scale
5. Build up your team and systems
This is an opportunity to take your enterprise to the next level, don’t let this modernization opportunity go to waste
Double down on what works, stop doing the things that aren’t producing results
Let’s get started…
Know your inventory
The more you understand what you are currently maintaining, the more you can understand project needs and risks.
Database transformations can be complex and resource intensive. They may require:
- Various teams and personnel
- Comprehensive migration and implementation plans
- Mitigating risk to mission-critical business operations
As such, agencies should know what tools and services are available internally, and when to apply them in developing an enterprise architecture-based database strategy.
Before you begin developing your database transformation strategies, it’s important to understand what’s incorporated in your legacy database portfolio and any complexities tied to that portfolio.
- To do so, assess the extent to which the portfolio currently meets your agency’s business needs.
The portfolio inventory and assessment will provide a snapshot of both the current and future environment, and the complexities associated with those environments.
- It should include: security, functional health, technical health, strategic alignment, and financial impact of the portfolio.
- Technological complexity should take into account information types, product-specific implementations, and the architecture of the legacy solution.
Examining these topics will provide the data-driven insight needed to make defensible decisions, reduce risk, and prioritize.
While these are all compelling reasons to transform, the key to a successful database transformation is to align the business value with the database transformation strategy and implementation strategy.
To preserve and structure the business knowledge and functions residing in legacy systems, your strategy should:
- Allow users to easily get actionable and unified information the way they want it, and at the speed their business needs it.
This will improve the overall efficiency and effectiveness in performing the functions that support mission priorities, and revitalize IT investments.
Having clear priorities will help you set meaningful goals.
Transforming “everything under the sun” can be highly risky, time consuming, extremely costly, and burdensome on resources. As a result, the scope of your database transformation should be well-defined.
Your goals might be a combination of:
- cost avoidance
Set achievable goals that move you towards your priorities. Focus on a few core goals and define them.
For example, modernization in the context of databases might mean:
- database code is in source control
- continuous integration and testing
- the underlying system can be patched quickly
- database code is documented
Once you define your goals it’s important to establish well thought out metrics to gauge your level of success.
- IT modernization architecture should include defined performance measures and outcomes that support your organization’s objectives.
Clear goals and metrics will enable you to establish the project scope and roadmap of your database transformation. Be sure you capture a valid baseline of performance and network typology that can be used to compare the perfomance of the to-be architecture.
Once you have made a decision on your specific scope and have a good grasp of your database portfolio, you can start looking at projects more granularly.
The first thing to ask, for each project, “Do I need to directly maintain a database?” Investigate SaaS solutions and see if there is existing tools for frameworks that meet your goals.
When you determine a database migration will best support your goals, you can begin the database technology selection. This is one of the most important decisions when designing data-driven systems.
Database transformations allow you to transform to more modular systems:
- Avoiding monolithic systems and proprietary technologies with shrinking talent pools
- Improving business value to customers
- Yielding operational efficiencies
- Enabling simple integrations with other systems
- Lowering total cost of ownership associated with licenses and maintenance/sustenance costs of legacy environments
- Providing greater agility, resiliency, scalability, and performance
In implementing a database transformation, you should start small and scale fast. Use an agile approach to reduce risk and drive quality. Deliver modernized components incrementally, transforming the bare minimum necessary to:
- deploy faster, obtain feedback from users, create technical architecture validation.
Build Up Your Teams and Systems
Build Your Team:
A robust workforce is the key to any successful business transformation. In-house talent is especially useful for database systems, since data persists longer than individual contracts.
Make sure that your modernization plan involves training that exposes people to new technologies early. When new systems and tools are chosen, make sure the people that will be developing and maintaining those systems have sandboxes to test the new technologies- this could be on their own laptops or virtual environments. Send FTEs to get relevant skills training as early as possible.
Consider using this training as an opportunity to instill best practices so that the teams are modernizing along with the systems they support. Set up source control for your database code, which is any code that interacts with the database. Make sure the assets in the system are documented and that documentation is maintained. Leverage automated migrations. Evaluate that your operational security is up to standards.
Making sure your team is supported during a transition will make managing the changes easier on them. Likewise, support from key leaders will give confidence to new processes and dissuade people from resisting changes.
Build your systems:
After you understand your current systems and have clear priorities and goals, building your systems will be much more straightforward and meaningful.
For your pilot project and subsequent projects make sure your solutions fit the goals and needs.
Choosing a databases to fit your goals:
A database is an intricate part of an application. As such, changing the database technology may require changes to other layers of the overall application.
- For example, there may be refactoring and/or re-engineering of your system’s backend.
To make the most informed decision when selecting a database technology, you should know what your **database criteria **are.
Considerations and Requirements include:
- Robustness and reliability
- Operational and querying capabilities Availability
- Amount of data
Type and structure of data
- Talent pool and availability of relevant skills
- Database vendor support funding, stability, and community
- Level of establishment
- Cost of licenses
- Ease of data transfer
- Easy to patch and maintain
- Good documentation for future development
- Infrastructure costs
- Platform specific features and limitations
Which Database Is Right For Each Project?
It is normal to have more than one database technology in satisfying your database needs.
- Certain databases are popular for their speed and scalability, while other databases are preferred for being highly structured.
You should understand your specific use case in selecting the right database technology.
No matter what database or service you use. Make sure you own your own data.
When it comes to database technologies, there’s no one-size-fits-all solution. Although different database technologies serve different tasks, your database technology should support all data types - structured and unstructured.
To this end, consider open-source database technologies to “future proof” your strategy.
Open-source databases are more modular, promote open standards and APIs, and extend foundational elements essential to:
- Avoiding a one-size-fits-all feature set
- Shifting from low-value work to high-value work
- Robustness of code
- Flexible and scalable platform for the future
- Scaling up or out without direct cost/license restrictions
By implementing open-source databases, you can yield a lower total cost of ownership, and shift the cost burden from upfront licensing to implementation and customization.
Also consider any encryption requirements when selecting a viable open source database engine. Applications with Personally Identifiable Information (PII) or Payment Card Information (PCI) data may have Transparent Data Encryption (TDE) or Cell-Level Encryption (CLE) requirements. Thoroughly research which encryption features are supported by the to-be database engine.
Maintain control over quality by combining your iterations with incremental releases, early prototyping, and testing.
Leverage the phased approach below in executing incremental funding and incremental development.
- Phase 1/ Proof-of-Concept:
- Identify prototype candidates, and test various database records to see if a target technology meets your specific needs.
Phase 2/ Pilot:
- Pilot a database technology with pilot candidates to uncover legacy technology obstacles, increase confidence, and gain user buy-in.
- Evaluate the success of your pilot based on the goals and metrics you established.
- Report out on what strategies worked and what experiments failed
- Phase 3/ Scalable Execution
- Fully execute your selected database transition - this may go beyond just transitioning the database, but application transformation. Have a detailed design, building, and configuration of the new environment, system, and application.
- Test to ensure all requirements have been met.
- Deploy the new database solution.
- Continue iterating
- Apply what you have learned to new projects
- Continue to Test, Deploy, Patch and Maintain your solutions
Database Modernization Case Study
Database transformations always require communication and agreement at the top levels. Without leadership buy-in, teams may push back due to other competing priorities.
When GSA’s Identity and Access Management System reached the end of its lifecycle, GSA needed to begin a database transformation. All the impacted executive system owners were involved and on-board before diving into the work. With top-level buy-in, integrated security, application and business teams were able to organize in DevSecOps structures and were also able to assist and participate in a smooth transition to the new system.
- Communication from the executive application teams to customers is essential for impact, changes, and future direction.
- Expect push back from certain contracts and contractors. Including extra time early on in your contracting process will help mitigate potential problems down the line.
- Handle risks and change management through the normal process rather than introducing a new process to the mix.
- Regularly communicate with application directors and executive system owners.
- Too many emails and meetings can become overbearing and inefficient.
- One-on-one chats can be forthcoming, and provide both good and bad insight to the process.
- Don’t forget the closeout activities such as updating ATOs and properly decommissioning any hardware or you will be answering for it in years to come. It’s not as critical but it still needs to be completed.
- It’s better to tackle a database transformation earlier in a system’s lifecycle than later.
Case Study Update
GSA is wrapping up the Pilot phase of a project where we are partnering with the TMF to modernize our application and database infrastructure: Application Modernization Integrating Flexible Architectures. Please refer to the link for the details and goals of the project.
We are updating the playbook based on our experience from this project to ensure we are providing teams with lessons learned and best practices. The team will host retrospectives later this month (August 2019) once we closeout Pilot Phase deliverables. Thanks and stay tuned!
“Boost” by Dev Patel from thenounproject.com; “Bow and Arrow” by faisalovers from thenounproject.com; “Archery” by Oliviu Stoian from thenounproject.com_; _“Flask” by NOPIXEL from thenounproject.com; “Telescope” icon by Eucalyp, from thenounproject.com_; _“Sabotage” by Krisada from thenounproject.com; “Integration” by Tokka Elkholy from thenounproject.com_; _“Delivery Man” by Виталий Плут from thenounproject.com; “Jet Fighter Pilot” by Gan Khoon Lay from thenounproject.com_; _“Rocket” by Nikita Tcherednikov from thenounproject.com