1. Global architecture

Planet 4 is built on individual instances of a very common and powerful Content Management System (CMS): WordPress. Here’s a snapshot of how the global architecture is designed:

High level multi-instance architecture for Planet 4
High level multi-instance architecture for Planet 4

Planet 4 is being constantly iterated on using open and agile production methods. This means that we are releasing early and often to regularly receive feedback from the community and incorporate those changes into the design.


2. General hosting setup

The hosting of the P4 websites happens on Google Cloud Projects. Google Cloud Project was chosen for its ease of automation, and because our hosting there is 100% from Renewable Energy Sources (RES). For each environment all three sites (develop, staging and production), the media assets and everything else will be hosted on Google Cloud servers.

All our automatic implementation has been built on tools and processes offered by Google, but could be deployed in other solutions that support Helm/Kubernetes (with the necessary modifications).

All our deployment code and scripts is in open repositories in github.

All our sites are built and deployed using CircleCI (as our continuous integration server), docker images.

NROs that have developers are getting the necessary permissions and documentation to deploy new code in their online environments. Currently (as of November 2018) the Netherlands and Luxembourg have made modifications to their sites (by modifying their child theme and developing extra functionality via custom made plugins)


3. NRO sites and environments

Once the Google Cloud Hosting is created, the GPI sysadmins / developers will create three sites and give NRO teams access to:

KEY INFO – Every 1st of each month at 01:10 am CET, a script will automatically sync the database from Production > Staging and Production > Develop. There will not be syncing the other way round though. So, changes in staging and develop will be overwritten. Changes in production will never be overwritten. 

This process affects ALL Sites, even implementing ones. 


4. The infrastructure

Google Kubernetes Engine (GKE) is the SAAS Kubernetes platform of choice, ensuring optimal balance between configurability and ease of use. Configuration described via Terraform (here in github).

Infrastructure monitoring happens via NewRelic Infrastructure deployed as a Helm chart (here in github).

Traefik ingress controller managing path-based and domain routing to Kubernetes services, and automatic LetsEncrypt certificates (here in github).


Base Containers

Containers are intended to be a foundation for future projects, and are not purely Planet 4 specific. They are intended as common platforms from which to deploy infinite websites with identical infrastructure but differing content.


NRO Containers

NRO Generator

Initiates a new NRO repository, Stateless buckets, CloudSQL databases and CircleCI project. Triggers initial deployments to develop and release environments.

NRO Sites

Each NRO is a new repository, which is built from the NRO Generator repository template. it consists of three distinct environments: develop, release and master, each of which is a distinct set of resources and its own Helm deployment.

Example: India


Testing

https://github.com/greenpeace/planet4-docker/tree/master/tests

Tests are written in Bash Automated Testing System (BATS), a low-level test framework for checking bash operations and scripts.

On each commit the containers are built, tagged by commit SHA, branch and/or tag and run through unit/integration tests. Successful test suites on master branch are then tagged with ‘latest’


Deployment

Helm package manager is the Kubernetes deployment tool of choice and recently became the native packaging solution.

Commits to NRO repositories trigger pipeline build/deploy workflows on CircleCI.

Develop / release branches are automatically deployed with Helm.

Tagged releases require manual intervention before being deployed to production.


Git Automation and Environment Promotion

The P4 Git Repo Dependency. Arrows indicate build hierarchy

CircleCI Workflows

Builds on upstream repositories trigger Circle CI workflows on each branch, with the following general format:


Terraform Infrastructure as Code

GKE and CloudSQL is described by the private github repository https://github.com/greenpeace/planet4-terraform-infra, which references the public repository https://github.com/greenpeace/planet4-terraform-modules

Terragrunt is a thin wrapper around Terraform, which encourages DRY, modular code and automates remote state synchronisation.

IAC Change Management [WIP]

The master branch of planet4-terraform-infra will be locked and require pull requests from develop. This will permit change reports from `terragrunt plan` to go through peer review before being implemented in CI.


5. Local Development

Anyone who wants to do development will have to use their own local machine, by building our development environment. The approach requires you to install docker and following the instructions.

The instructions have been followed and worked for many people until now, as this is also the environment that all P4 engineers are using. But if you hit a bug or find parts of the documentation missing or inaccurate, please either open a ticket or open a Pull Request to the relevant repository.

If you are in an NRO, you should also check out this document with information on how to get in touch with the rest of the p4 dev community.


6. Coding Standards

We have chosen to follow WordPress’ code standards, styling and best practices.

More information can be found in the Planet 4 development page.

NOTICE: We expect any developer to have read and follow these before they start working on P4 code.


7. Code Repositories

(in case you want to contribute)

We use GitFlow as our methodology for version control (branches, naming conventions, how to work when every part of the application is in a different repository etc).

The code written by the planet4 team that makes the planet4 websites have the functionality and the design they have is controlled by github repositories. (Description for each exists in the plugin page). The major of those are:

Planet4 Master Theme

Planet4 Child Themes

Planet4 blocks

Planet4 Medial Library

Planet4 Engaging Networks


8. Plugins: list and review process

The list of currently used plugins can be found on the Handbook Plugins page

Installation and updates of plugins is happening via the composer scripts. All the plugins that are present in all installations are defined in the common composer file.

Additionally, plugins that are installed only on a specific P4 site is defined in the composer file for that site. For example, Loco Translate is only installed on the handbook site, and is defined in the handbook composer file.

The process for adding new plugins is also described in the P4 Plugins page


9. Release: the process

Currently the process for releasing new code can be summarised in the following

  1. Issues (either bugs for fixing or new features for development) are being added in Jira
  2. Product owner prioritizes those issues
  3. At the beginning/planning of each sprint (1 week) the development team picks up the issues with the highest priority
  4. Development happens
  5. All the code passes peer review (via github pull requests), and is only merged in the master code once it has been approved by a second engineer.
  6. Every Monday, a UAT (User acceptance testing) meeting happens.
  7. If things don’t “pass” the UAT, the developers continue working on it until they get passed. If need we have a secondary UAT meeting on Wednesdays.
  8. If everything that has been developed until then gets approved, we are doing a release and we create a page describing the changes

10. Release: the deployment

For code to be deployed in an online (staging or production) environment, the code needs to be in a correct git tag based on the master repository, like the ones below:

Then the relevant composer.json file is changed so that it references that new tag. For example going from version 1.13 to version 1.14 included changing the tags of master-theme and plugin-blocks:

This roughly creates a circle of doing new releases once per week. But we are not doing releases just for the sake or release, and we are not going to rush a release if we are not very happy with the code. Additionally, some times features are quite big and they span a lot of sprints (weeks), which could result in a week not having a release, or having the release on a Wednesday or Thursday instead of the Monday. Doing correct releases is more important than doing releases on specific dates.

This means that an issue (that is not deemed “urgent”) gets filed for example today, even if it goes to the top of the list, it will not be live for at least two weeks.

Urgent bugs:

If the product owner considers something really urgent, we have the ability to add a bugfix unplanned inside the work of a sprint (week). This is generally disruptive for the work that has been planned, and thus we are very very careful with what is considered “urgent”.

Detailed release process (for Virtual Machines or K8s):

Check how to prepare a release (for DEVs only)


11. Watch the Tech Intro video

Here’s a video to better understand the technology behind P4, with the unique voice of Master Konstantinos:


12. Implementation FAQs

Have you checked the Planet 4 Implementation FAQs? There’s a lot more info you may be interested.


Links & Resources