Prototyping a New Project: Challenges and Solutions

About the project

We started the collaboration on this project in the early phase of the actual development, even though some of the base logic like authentication, roles, some placeholder API endpoints were already implemented.

The team at that time consisted of 3 people working on the client’s side, and 3 people joined gradually from our side.

The technical background of the application is a PHP backend with Symfony framework, MySQL database, Redis cluster, a RabbitMQ for the messaging and some other small services running in a Kubernetes environment. It’s always fun to start working on a project like this, because on one hand we can use a lot of different types of technologies, and on the other hand it is a huge challenge to keep it fast, secure, and well-designed until the end of the development.

Challenges – the good ones

The scope of this project and the technology stack is exciting for a developer, especially handling different types of data sources, processing huge data files in the required time frame. These challenges need non-standard or out of the box complex solutions.

Challenges – the not so good ones

The first challenge is not a huge one, but it can be a pain sometimes: when the team started to grow, we had to design and agree on branching strategy to be able to manage the source code properly. It was needed to avoid a lot of merge conflicts and “cherry picking” features during the development to save time. When a smaller team was working on this project with 1 or 2 developers, it was not a problem, but after we joined the team and the number of the merge requests started to increase, the tech lead had no time to review these on time. As a result, it started to cause a lot of extra work because of the dependency between the requests.

The solution is to delegate some of the responsibilities, so that no one would be too busy. We had the trust of the client, so now we are reviewing each other’s changes, so the development can go faster, and the tech lead could work on what is most important: understanding the requirements.

The second challenge is much bigger, and it has a lot of side effects, that cause other challenges. As I mentioned, this project is now in the prototyping phase. It means that a lot of requirements are not done, missing, or constantly changing.

We can say that it is totally fine, because as software developers, we are used to it, and this is the core of the Agile method, but changing the data model takes too much time, because it affects the entire application. Unfortunately, we have no time to step back and finalize the whole data-model/architecture, so we always need to refactor a bunch of business logic.

The other side effect of this type of development is that many times when a requirement has changed, for the small issues we said, “we will fix it later” and the list of these “fix it later” issues are getting larger and larger. Unfortunately, we faced such problems in the past: everyone thought that a feature has already been implemented, and when we wanted to use that feature, we realized it wasn’t. We do not currently have a silver bullet solution for this issue. Dedicating more time to understand the requirements and working with the end clients much more closely to clarify that constant functional changes affect not just the deadline, but the software stability as well.

Summary

The velocity of the development could improve, but we are steadily making progress and we definitely see, that the team members are doing their best to complete the project successfully and avoid as much pain as possible. But in the end the most important thing is to bring some golden value to the clients.

So the conclusion is: Software development is hard but we enjoy it.