In my career, I developed applications from scratch multiple times. I managed team for enterprise platform, started 5 years ago and have to engage my team, think and plan for application modernization, leading to this blog.
Assume platform was written in old language (Example: PHP) and leverage open source frameworks(Drupal or CodeIgniter or Laravel) and team has thinks that the application needs to be rebuild. Re-builds are always longer, harder (than we think), and filled with peril. The ability to maintain the discipline required for a re-build across long duration is a rare commodity.
Remind that developer developers are an optimistic bunch when it comes to how long it will take to rewrite and tech stacks will change over time. Focus on business value, architecture and approach to build. Even with experienced developer, it may take far longer than anticipated. here are some questions I had to think for answers
- How to convey and get the business to really understand what it will take to re-build a large complicated web application that has years of code, business logic, etc.?
- If we are going to rebuild a new web application with a decent amount of financial data, what languages/DBs would be pragmatic choice?
- Do we rewrite in PHP? Will firms be able to recruit PHP developers? Is it worth switching to a different language to rebuild? Switching means hiring new staff to do the reb-build, while it may be short sighted to decide PHP without pragmatic evaluation .
- Do we consider building from scratch, again in PHP and related PHP frameworks? Will writing in place be probably easier than starting over from scratch?
- Do we build a new application and roll out the new one separately? I like to get new code into production as soon as it’s ready, while that may give rise to compatibility challenges and end up taking longer time.
- Should we hire an outside firm to do a rebuild? When deciding technology, be pragmatic of the company’s ability to build a productive team, in addition to tech stack. As languages age, team dynamics that work on language change. Mature technologies tend to have more mature people working, and the youngest (at heart) engineers move on to the latest toys. Are you in a stage where it is harder to find people who want to work in language full-time?. This can be solved with outsourced companies that specialize in them. The number of PHP developers is not small, but the number of qualified ones is.
Now on approach. May be start with a rebuild MVP and then iterate development. A smaller version at a smaller cost to engage the business before the whole thing is “done”. Be aware of the large risk of running out of momentum/money before it’s done
Migration is more of a business decision than a technical one. before decision, start with a deep dive on the business side to truly understand what underlying problems are being solved. Better to capture insights from code review, insights from enriched bug lists with severity and priority, , document known system limitations, security and performance concerns and UI/UX concerns. Invest effort effort to perform a value proposition mapping exercise to understand client needs and build a powerful business case.
- Is a really old codebase used, or is existing code developed on a framework that is dead or no-longer-maintained?
- Was the code base developed with very little oversight and less skilled people working on it for years, leading to a “ball of mud”? Is there more hard-coding (say hard-coded SQL)? How ready is existing code for reuse? Has overall app maintenance been a source of struggle?
- For a company in a growth stage, innovation is important. If going back to PHP, Can you recruit people who want to perform ground-breaking and/or new product innovation?
- Has application UI/UX aged as per business and customers? A real reason that business will consider to invest in revamp and transformation of UI/UX of the product for the next 5-7 years of its life
- Are there custom solutions? Why are they important today? Are those IPs worth building again? is it core to their unique competitive advantage
How much team and product has grown in the past few years? An accidental SaaS platform may start with few engineers and few features in attempt to solve a problem. Then teams started to build around them over years like tentacles of spiders. Start looking at the current team and map back to the beginning. Look for hidden business logic in code. It is quite possible for one to think one knows what needs to be done, one may not. Be prepared to rediscover things as you move forward. A pessimist thumb rule is: if it took 8 person years to build, it may take 2-3 person years to rebuild.
Perform code review in system components that are likely to be in demand for features in the future. How easy it would be to change these components? If components are discrete and decoupled from monolith, they may be easy to migrate to a new interface, with assumption, automated tests are also in place.
Other options to consider are upgrade feature by feature and keep the database the same to start with and then finally upgrade the database. Remember that while internal facing features can be updated based on iterations, externally, businesses may want a clean release with a single change in the user experience for customers.
To reduce risks, businesses may want development to consider rebuilding in chunks rather than big bang new building( all at once) to gain a much shorter feedback cycle. Take an approach where there is potential for less number of unexpected bugs. More unexpected issues cause dissatisfaction of business executives who feel “keep waiting and waiting” to get back to where they were before and can become political across departments. Let us keep in mind that the market would continue to change requirements for the new system as the same is getting developed, leads to even more further delay for demonstration of value to business
Has the core value of the product changed?
What is the core value? It is possible that there is no need to rebuild the current product and we need to build a new product that meets today’s needs. Can you carve that part, offer a new version and migrate customers across? If business has changed significantly, it may be a better option to build a new product that does what customers need today. This provides more leeway to development team to make technical choices, while the current product is put in to maintenance mode.
Remember that building a brand new system with tons of new features may take a long time to build, and then migrate. In this context, one can develop new front-end with a combination of new and existing API, maybe proxy/delegate, especially for new modules and keeps users engaged and has a better outcome from business perspectives
Microservices solve scalability problems, not complexity problems. While services are great, 100s of services create a hard problem to fill engineer roles (if that is what you want, go ahead). While software-defined networking is simpler than physical networking, it is not an skill for developers (who do not already have it ) to ramp-up and learn and may be difficult to hire, opening to downside potential to increase both the application complexity and the surface area of security-vulnerability. Unless micro services solve existing problem, please question rationally whether micro service architecture is required, micro services are much harder to build, secure, and maintain than monolithic architecture
If the plan is to change tech stacks and include things like micro services, it may be worth to plan a stress-test of current platform by engineering team to arrive at performance benchmarks from user perspective. Everything can potentially change from runtime characteristics to build tools to deployment models. Benchmark the time to complete the same task on each platform and extrapolate from that. This can be a way to differentiate between hype and reality at MVP level. Do not agree with developers claim “it runs great on local”, drive team to get it run as part of daily workflow in production, measure performance and usability, compare with the benchmark.
Think of health of infrastructure to scale with more users, and health of the code to build more features with confidence. With the advent of cloud, infrastructure has become easier than earlier. It is easier to move from bare metal to scalable groups with sticky sessions or sessions in a database and also easy to commission and decommission MySQL instances in AWS
Are core application challenges in the existing application related to specifics of your existing stack? If yes, please consider most similar tech stack to existing one that resolves those specific issues. Any new, radically different technology stack has potential to slow down dramatically and result in a huge amount of technical debt, make the product worse. Different stack may lead to higher cost to pay, as differences in language syntax and ecosystem may be different and non-trivial(change paradigms such as OO->functional or event-driven/async)
- Start it off with a small team of maybe 1-3 very experienced engineers, and let them be focused on the work. Treat it like a startup of a small team to do one thing and do it well, and you’ll be surprised how fast they can move.
- If an application has components for data aggregation, API, and a web app to handle, splitting the approach to address them is a good first step. In this case, address the web app first
- Migration of features to new architecture with less experienced folks or making the same team perform migration alongside maintaining/extending the existing product is super hard.
- Tacking early stages of a large green field project with a large team is also super hard, making the project consume more time than was estimated.
- Attempting to make the new system work with the old one is a worthwhile exercise. It’s the worst of both worlds. You are limited by most of whatever was wrong with the old system – then you’re not really writing a new system, you’re just updating it.