Software prototyping and ideas coming into life

There is nothing perfect in Software. I always think about how things could have been done better and sometimes in our profession we tend to over engineer work that is already there. I think this is a double edge case where you can get lost in perfection or you work on something that does not convince you.

I had both feelings when working with some projects and this post is about some thoughts I had that I wish someone had told me for quick prototyping and weekend ideas.

Balsamiq: tool for prototyping

Balsamiq: tool for prototyping

This is a list i will keep expanding conforming I work on more and more projects in my free time.

1. Just keep it simple. Don’t over engineer it

As software engineers we tend to over engineer and think about our idea will be used by millions of people, which might be true but at that point you will have the resources to think about these problems, or even better, you will have someone who is specialized in that area and can do the work better than you.

My idea is normally, just make it simple, easy to scale (at least for the first 1k users/views) with not a lot of dependencies between systems or complex data sharding strategies. Use something you are comfortable with and make it happen in less than a day, at least the concept.

In other words, the KISS principle. This relates in my case to:

  • Use a technology you are comfortable with
  • Talk about the idea with a non-tech person. Find out a way to showcase it with a sentence and code only that idea. At the end of the idea, validate it again with that person.

2. Write a generator

Every time you have a new idea there is a lot of code that is boilerplate. At least in Java (my main programming language) although I would say it happens in every language and stack: metrics exposure, web, service and data layer, management configuration, security (at least authentication, authorisation), project structure, readmes…

There are so many files, so many jobs and so many classes that can be automatically generated. For example, if the application supports oauth, having the sql, the class that  retrieves it, the grant types, roles and rest layer setup could help a lot when starting a new project. This, in my experience is always the same, so why write it again?

The disadvantage of it is that for someone that does not understand what is going on under the hood could be a bit complex on the first step. I recommend to use a generator (using existing ones or your own one, but understanding the code created).

Approach:

I use yeoman as a code generator.

For a quick prototype i normally go with node with a touch of angular using angular: angular-fullstack. Although recently I also used react-fullstack and I am quite impressed with the ease of development.

For a more mature idea that normally involves a bit more heavy processing I choose JHipster to generate the backend.

Use of Yeoman for structured code generation on new ideas

Use of Yeoman for structured code generation on new ideas

3. Avoid complex data access retrievals

Meaning, try to keep it as simple and understandable as possible without getting complicated even if it is the right way to do it.

One example is the use JPA instead of JDBC for example for accessing to data. Almost all my professional career, I have worked with plain sql and controlling how to access/retrieve/cache the data retrieved from database. The main problem is that it requires time to write all the repositories for the models and normally it is not the main core of your business at the beginning. My thought is more towards keep it simple, everyone knows JPA, not a lot of people are efficient with SQL, which is a completely different world. If the business requires performance, you always can use a jdbc template for a specific data access implementation.

In my mind, for prototyping, you should not spend lot of time normalizing, adding sneaky indexes and figuring out caching strategies. Chances are that you can support by default thousands of users and if required a project that took a weekend can be refactored on another weekend if got enough acceptance. Chances are that 90% of the times, it is not, so In my mind it is saving time.

Approach:

  • I use RDBMS instead of NoSQL technologies normally as my data is normally structured.
  • Prefer the use of JPA with hibernate cache instead of JDBC with custom caching
  • I try to avoid comple data structures. This will come later once the prototype has been validated.

4. Expose metrics

When coding an idea, we are not going to have all the indicators for growing the business. However, we could use an out of the box solution to expose some details we may be interested such us:

  • Number of users registered
  • Number of logins
  • Number of subscriptions
  • Endpoint calls grouped by name
  • Memory
  • Health statuses

Approach:

I use actuator on spring boot normally. It is an out of the box solution for standalone java applications. It lets me register the metrics I am interesting with the MetricsRegistry class, and through an exporter, have that data exported to a stats receiver (Ex: elastic search, influxdb, graphite…).

You can do that manually as well through a async http connection to your stats receiver, it is just more complex software to maintain. Hence, I normally rely on spring boot.

Metrics architecture with ELK

Metrics architecture with ELK

 

Although, I had as well really good experience with Grafana (dashboard) and Influxdb (time series db for data buffering) as well for representing information. An example of the dashboard is here:

Grafana dashboard for metrics monitoring

Example of grafana dashboard use for metrics monitoring

Separate frontend and backend

For a quick prototype (day hack or weekend), I try to keep a project separating the frontend and backend. This is normally becoming the standard (at least separation on folder structure). I try to go a bit further and to separate them into different repositories or have them completely isolated. This could add a bit of overhead but In my mind it is a good practive to keep them separated.

Reason: scalability

Main benefit with the separation is that you could use both in different processes and scalate them depending of their needs: high I/O on the backend will not imply increase the number of machines on the web servers. Heavy load on the web servers will affect only on adding more machines on the web server layer and keep the backend as it is. We could put the money where needed.

Reason: team collaboration

Chances are you will work on your idea with other people. I tend to work with the backend while other friends/coworkers work with the frontend. The communication between the backend and the frontends is normally through an API.

Reason: Extensibility

The idea is going well? If the frontend and backend are separated, you could extend the idea to mobile writing an app that has a client to communicate with the rest api exposed on the frontend.

Frontend, Backend architecture proposed on Amazon

Frontend, Backend architecture proposed on Amazon

This is on a complete subject which i will try to cover in the future. The conversation between frontend – backend on same project, different ones and so on has been discussed a lot and everyone has its own point of view.

Summary

This post is just a organization of ideas that I normally have learnt/investigated/have in my mind when creating projects. Writing them down and make it public is my way to see what other people think and extend the list in the future.

Leave a Reply

Your email address will not be published. Required fields are marked *