Going n.U.T.S - Developing with the nerdgeschoss Unified Tech Stack

When developing applications quickly, you can’t afford building everything from scratch every time. This is why we came up with nUTS, our tech stack that focusses on

  • using open source software
  • make the right things simple and the wrong things hard
  • reuse existing code. No one needs yet another implementation of async modal windows.
  • maintainability. Follow common best practices and make it easy for other developers to take over a project
  • developer experience. Make it pleasant to work on a project and easy to get started.

What’s in it for me?

Customers

With reusable components your project gets off the ground quicker. Instead of weeks of infrastructure and project setup, this tech stack comes out of the box with a production ready deployment path, review apps and security settings. Instead of rewriting how modal windows work and how to login, we can focus directly on what matters: Your business.

Designers

Tired of discussing with developers why a 1px letter spacing is needed? This tech stack lets you work with design tokens that directly translate into code, usable by developers. Our system for vertical spacing using Figma Auto Layout helps you create easy to maintain designs that are also easy to hand off to developers.

Developers

Solve problems, not tickets. Reimplementing the same feature over and over is just time consuming and boring. With this setup you get to the actual business logic faster. Apart from that, switching between projects becomes easy. Run bin/dev and you have a project running on your machine in seconds instead of hours; context switching becomes a breeze.

Development Setup

Getting your Mac ready for development

First you need some software installed that other tools will depend on. Make sure your Terminal (or iTerm) is running under Rosetta 2 and Homebrew is installed. Also install Docker for Mac. Then install dependencies (refer to the troubleshooting section at the end of the guide if you run into problems).

brew install postgresql@15 mysql rbenv tmux overmind imagemagick
npm install -g yarn

Clone the development environment project and start the databases (the included docker compose file contains all versions of databases and dependencies we need between projects):

mkdir ~/Developer # if it does not yet exist
cd ~/Developer
git clone [email protected]:nerdgeschoss/development-environment.git
cd development-environment
docker-compose up -d # this command needs to be executed after every system restart

You can install a new ruby version via

rbenv install 3.1.4 # replace by desired version number, see .ruby-version file

Starting a Rails application

After cloning your project, run these commands to set it up:

bundle # install ruby dependencies
yarn # install JS dependencies
rails db:create db:migrate db:seed # makes sure you have a database with some seed data

Notes

Some projects do not have seeds and use fixtures instead. It should be listed in the readme of each repository. For all future projects we will use seeds. If there are no seeds for a project you can run:

rails db:fixtures:load

Then, to start your project:

bin/dev # will start all necessary components of the app within overmind

Obtaining Project Secrets

Projects often require specific secrets or keys for access to databases, APIs, or other resources. In many of our projects, we manage these secrets through Rails Credentials, although some older projects may still use .env files.

When you add a new secret to a project, it should be inserted into the credentials file. This is achieved using export EDITOR="code --wait" rails credentials:edit. For uniform access, our company gem, shimmer, pulls these keys using Config.#{ENV_KEY_NAME}. This approach ensures compatibility with both .env and Rails encrypted credentials files.

To access the credentials file, you need the RAILS_MASTER_KEY. You can find this key in the environment variables on the Heroku server. Be sure to handle this key with care due to its sensitive nature.

Debugging a Rails application

If you ever need to debug a Rails app, you can use debug or pry (in older apps) like this:

debugger

respectively

binding.pry

Execution will stop at this point giving you a chance to connect to the tmux session via overmind:

overmind connect web

To leave the connected session, press [ctrl] + [b], followed by [d] (hitting ctrl+c would kill the session instead).

You can also restart the application (without completely restarting the whole stack) via

overmind restart web 

or

rails restart 

Creating a new Rails application

rails new YOUR_APP_NAME --database=postgresql --skip-jbuilder --skip-test --javascript=esbuild --skip-bundle --force --template=https://raw.githubusercontent.com/nerdgeschoss/development-environment/main/rails-template.rb

Tools

Services

Heroku Sentry NewRelic Skylight

Gems

This is an incomplete list of gems (see the Gemfile of the project generator for explanations of each gem) highlighting only the most important gems.

shimmer A collection of reusable stuff in our applications, from image management to modals and popovers. See it less as a library and more of shared code between projects. It’s recommended to read the whole source code of the gem before using it.

puma Our application server of choice. It works nicely with Heroku and has a good performance. Might be replaced by a fiber-based server like Falcon in the future.

sidekiq + sidekiq-scheduler Background jobs are executed via ActiveJob, using sidekiq as a backend. This comes at a small performance cost but makes development a lot easier. sidekiq-scheduler allows for scheduling cronjobs without any additional dynos.

slim Slim is our templating language. It allows for short, concise syntax and really shines when dealing with complicated conditional classes and attributes.

pundit Authorization is channeled through pundit. Thanks to the tight integration with avo, we can reuse policies between the app and the admin panel.

yael Yael helps you to emit events and react to them in the background (e.g. send a welcome email after a user signed up) within the existing rails setup (postgres + redis, no need for Kafka). It’s also a good logging solution for important things that happen during object lifecycles (e.g. password changes, failed login attempts, …)

avo (Pro) Avo brings a responsive admin interface that’s really easy to start and customise. We purchase a Pro license for each project to use the pundit integration and support for reorder-able lists.

rspec Testing happens with RSpec, focussing on model and system tests (see more about that in the testing guide)

capybara + cuprite to run system tests within chromium based browsers. Cuprite is a lot faster than selenium while offering the same (and more) features, also comes with all necessary dependencies out of the box.

vcr + webmock For stubbing web requests in tests. You don’t always need vcr, most of the time stubbing manually via webmock is preferred.

annotate used to annotate our models and specs via bundle exec annotate --models.

chusaku used to annotate our controller routes.

letter_opener Opens emails during development inside of the browser instead of just logging.

pixelpress allows easy creation and serving of pdfs.

faraday for network requests to external APIs.

graphql If we provide an API, we always do it as graphql. We use this gem to quickly generate the api, usually in combination with graphql-persisted_queries (for caching support) and the fiber based loader that comes out of the box.

sentry For error monitoring we use sentry on our projects.

capybara-screenshot-diff It allows us to compare screenshot in our system tests and we integrate this with our in-house software screamshot to compare diffs between PR's to prevent view regressions.

kaminari Our pagination library of choice. It's simple and just works for most use cases.

rubocop We use rubocop to standardize our codebase. The company configuration can be found in shimmer.

i18n-tasks Translations are managed by i18n. i18n-tasks health on all projects should be green which makes sure there are no extra/missing translations and that they are in alphabetical order.

cuprite We prefer cuprite over selenium in all our system tests. It's lighter-weight and faster than selenium.

Project Structure

We aim for monorepos with the monolithic Rails app at the center of everything. That’s why the app lives at the root of the project, containing Procfiles for development and all the settings for production and review apps (check the template for details).

If there are any extra things (e.g. native applications) they live in subfolders of the rails app (e.g. native/ios).

The heart of our project are server rendered applications, updated via partials on ajax/websocket requests (see https://hotwired.dev for details). On top of that we use the Remote Navigation feature of shimmer so we don’t have to write Turbo tags manually anymore.

Native applications are implemented either via Capacitor or if the project constraints allow it via Turbo Native.

Coding and Designing

Rails

  • Model hooks are evil. They have a tendency to fire in situations where you don’t expect it (especially when manually debugging or fixing things in production) and usually trigger some really embarrassing side effects. Instead, create specialised methods on your model that do the update and also trigger the side effect (or maybe emit an event via yael if those things are not directly related).
  • Mass assignment in controllers quickly turns into a mess. It’s ok for simple cases, but if your use cases become more sophisticated (e.g. updating multiple models, trigger side effects, send notifications, log audit trails, …) move things to a model method.
  • No where queries in controllers or models outside of scopes. Whenever you refer to a certain scope of records, actually use a scope. This helps maintainability when requirements change (e.g. what it means for an article to be “published”).

Frontend

  • Use BEM
  • One component per file, name like the component and inside of the components folder. All components are automatically required, you should never have to use an @import statement in your code.
  • Use css variables over scss variables. They can change at runtime and make theming a lot easier. If supported by the project, generate the variables via the Figma Tokens plugin.
  • Vertical and horizontal spacing of components is achieved by .stack. Inside of a component you may use margins if stack does not work for your use-case.
  • Frontend interactions are written as stimulus controllers in TypeScript. The linter should tell you about guidelines in coding them.
  • All images are either in webp or svg. No pngs or jpegs.

Trouble Shooting

MySQL

If bundle install fails with ld: library not found for -lzstd, follow the instructions of this post. Alternatively, try:

ls -la $(which mysql)

That gives you where the mysql binary is, something like /usr/local/bin/mysql -> ../Cellar/mysql/8.0.28/bin/mysql, meaning that your mysql install is in /usr/local/Cellar/mysql/8.0.28. Use that path for the next command.

gem install mysql2 -v '0.5.3' -- \
 --with-mysql-lib=/usr/local/Cellar/mysql/8.0.28/lib \
 --with-mysql-dir=/usr/local/Cellar/mysql/8.0.28 \
 --with-mysql-config=/usr/local/Cellar/mysql/8.0.28/bin/mysql_config \
 --with-mysql-include=/usr/local/Cellar/mysql/8.0.28/include

Postgres 15 Update Instructions

brew uninstall postgresql
brew install postgresql@15

# add this line to your `.zshrc` file:
export PATH="/opt/homebrew/opt/postgresql@15/bin:$PATH"
# if you're on x86 (Rosetta), it might be: export PATH="/usr/local/opt/postgresql@15/bin:$PATH"

# and re-source it
source ~/.zshrc

On any project you open you will need to run bundle pristine pg and bundle pristine annotate otherwise it will be using the postgres@14 adapter.