Feature branching in TM1

Enabling GitHub Flow in TM1

In the world of software, keeping track of changes is super important. In TM1 (IBM Planning Analytics), this used to be a big headache before Git came along. Let's see how things have changed over time.

Before Git: Doing Everything by Hand

Back in the day, managing versions in TM1 was a real pain. You had to do everything manually, like sorting files into folders and making sure you didn't mess up. Even something simple, like picking which files to include in a version, was a lot of work. It was tough, but people got used to it because there weren't any better options.

Enter Git: A Big Improvement

Around 2010, the introduction of a REST API endpoint marked a pivotal moment for TM1 users. Suddenly, the possibility of Git integration emerged, offering a glimmer of hope in the realm of version control. However, this was merely a technical possibility; there was no established best practice for implementation. Fast forward to today, and Git is fully part of TM1. Now, with a few clicks in the PAW interface, you can use Git without needing to write code. But TM1's REST API natively only supports environmental branching…

Branching Strategies: Environmental vs. feature branching (GitHub Flow)

Environmental branching and feature branching are two distinct strategies for managing code development workflows. Environmental branching involves creating separate branches for different environments, such as development, testing, staging, and production, allowing for isolation and controlled deployment of changes. On the other hand, GitHub Flow revolves around a simpler approach, where developers work on feature branches, submit pull requests for review, and merge changes directly into the main branch, prioritizing continuous integration and deployment. Each approach has its advantages and suitability depending on the project's scale, team size, and deployment requirements.

Environmental Branching

This method is all about making different branches for different stages of work, like development or testing. It's good for small teams and simple projects because it's easy to understand and use. But it's not great for fast-moving projects or big teams because it's too rigid.

Understanding TM1 GitFlow

At its core, TM1 GitFlow operates on the principle that the TM1 server is always tied to a specific environment branch. This ensures a linear evolution of TM1 development with every team member working on non-overlapping functionalities. And this connection is described in a JSON file ( tm1project.json) that acts as a set of rules that map the server state (schema and data) to Git, allowing for both general and environment-specific rules. It facilitates the management of object dependencies during push-pull operations.

GitHub Flow

With GitHub Flow, instead of making branches for different stages, you make branches for different features. This is good for projects that are always changing because it lets you work on different things at the same time. But it can be tricky for projects that need to keep track of lots of data, like databases.

Our Solution: Enabling GitHub Flow in TM1

Here's where our solution shines: we've unlocked the power of GitHub Flow within TM1. Despite TM1's REST API initially supporting only environmental branching, we've devised a method that allows you to embrace GitHub Flow seamlessly. Now, we can enjoy the flexibility and efficiency of GitHub Flow's feature-based branching strategy right within TM1.

Implementing feature branching on TM1

To implement feature branching we had to fulfil the following prerequisites:

  • We had to create independent and separated server environments for all developers where by default only the schema and the code exists
  • We had to separate the TM1 model into non-overlapping domains (schema, user/business data, control parameters/data), and create unique data management principles for each domain.
    • Schema: change only by code
    • User/business data: environment specific and handled by data pipelines
    • Control parameters/data: change only by code
  • (details on how we did that are coming soon in another blog post)
  • Based on these principles we had to define a client/project specific tm1project.json structure
  • We had to build a simple data management tool for our developers to easily manage test datasets between feature branch changes (see further details on that: here
  • We had to provide a simple tool for the developers to manage Git feature branches

Command-Line Tooling

To streamline the migration process and support multi-step branching, we developed a command-line tool, similar to the Postman libraries used for templating by the TM1 community. ThePowerShell script wraps the TM1 REST API and implements feature branching functionality. We use native Git so Postman was not an option for us. Instead we used PowerShell to write the script. This way we not just ensured cross-platform compatibility (Windows and Linux) but also easy integration with CI/CD pipelines (Azure DevOps, GitHub Actions, GitLab Pipelines, etc.). The command line tool can be used without extensive Git knowledge and it helped us a lot when onboarding team members who are not experienced in using Git.

This tool will soon be available in the KnowledgeSeed GitHub repository. So stay tuned.. :)