Dev Setup

☁️ Using GitPod

Gitpod enables development on remote machines and helps you get started with Middleware if your machine does not support running the project locally.

If you want to run the project locally you can setup using docker or setup everything manually.

1. Click the button below to open this project in Gitpod.

2. This will open a fully configured workspace in your browser with all the necessary dependencies already installed.

After initialization, you can access the server at port 3333 of the gitpod instance.

🐳 Using Docker

If you don't have docker installed, please install docker over here. Make sure docker is running.

1. Clone the Repository:

   git clone https://github.com/middlewarehq/middleware

2. Navigate to the Project Directory:

   cd middleware

3. Run dev.sh script in the project root 🪄
   ./dev.sh creates a .env file with required development environments and runs a CLI with does all the heavy lifting from tracking the container with docker compose watch to providing you with logs from different services.
   The usage is as follows:

   ./dev.sh

   You may update the env.example and set ENVIRONMENT=prod to run it in production setup.
   Further if any changes are required to be made to ports, you may update the docker-compose.yml file, accordingly.

4. Access the Application: Once the project is running, access the application through your web browser at http://localhost:3333. Further, other services can be accessed at:

   • The analytics server is available at http://localhost:9696.
   • The sync server can be accessed at http://localhost:9697.
   • The postgres database can be accessed at host: localhost, port: 5434, username: postgres, password: postgres, db name: mhq-oss.
   • The redis server can be accessed at host: localhost, port: 6385.

5. View the logs: Although the CLI tracks all logs, the logs of services running inside the container can be viewed in different terminals using the following commands:

   Frontend logs

   docker exec -it middleware-dev tail --lines 500 -f /var/log/web-server/web-server.log

   Backend api server logs

   docker exec -it middleware-dev tail --lines 500 -f /var/log/apiserver/apiserver.log

   Backend sync server logs

   docker exec -it middleware-dev tail --lines 500 -f /var/log/sync_server/sync_server.log

   Redis logs

   docker exec -it middleware-dev tail --lines 500 -f /var/log/redis/redis.log

   Postgres logs

   docker exec -it middleware-dev tail --lines 500 -f /var/log/postgres/postgres.log

🛠️ Manual Setup

To set up middleware locally, follow these steps:

1. Clone the Repository:

   git clone https://github.com/middlewarehq/middleware.git

2. Navigate to the Project Directory:

   cd middleware

3. Run Redis and Postgres Containers:

   If you don't have docker installed, please install docker over here

   Run the following commands to run Postgres and Redis using docker.

   cd database-docker && docker-compose up -d

   If you don't prefer Docker, you can choose to install Postgres and Redis manually.

   Once you are done with using or developing Middleware, you can choose to close these running container. (NOTE: Don't do this if you are following this document and trying to run Middleware.)

   cd database-docker/
   docker-compose down -v

4. Generate Encryption keys:

   Generate encryption keys for the project by running the following command in the project root directory:

   cd setup_utils && . ./generate_config_ini.sh && cd ..

5. Backend Server Setup

   • Install python version 3.11.6

     • For this you can install python from over here if you don't have it on your machine.

     • Install pyenev

       git clone https://github.com/pyenv/pyenv.git ~/.pyenv

     • Add pyenv to your shell's configuration file (.bashrc, .bash_profile, .zshrc, etc.):

       echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
       echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc

     • Reload your shell:

       source ~/.bashrc

   • Move backend directory to create a virtual environment

     cd backend
     python -m venv venv

   • Activate virtual environment

     . venv/bin/activate

   • Install Dependencies

     pip install -r requirements.txt -r dev-requirements.txt

   • Create a .env file in the root directory and add the following environment variables, replacing the values with your own if needed:

     DB_HOST=localhost
     DB_NAME=mhq-oss
     DB_PASS=postgres
     DB_PORT=5434
     DB_USER=postgres
     REDIS_HOST=localhost
     REDIS_PORT=6385
     ANALYTICS_SERVER_PORT=9696
     SYNC_SERVER_PORT=9697
     DEFAULT_SYNC_DAYS=31

   • Start the backend servers

     • Change Directory to analytics_server

       cd analytics_server

     • For backend analytics server:

       flask --app app --debug run --port 9696

     • For backend sync server:

       flask --app sync_app --debug run --port 9697

       NOTE: Open this sync sever in a new terminal window after activating the virtual environment only after starting analytics server.

6. Web Server Setup

   • Install NodeJs 16.17 (LTS) either manually or using a tool like nvm or volta.

   • Install yarn package manager

     npm install --global yarn

   • Change Directory to web-server and install packages

     cd web-server
     yarn

   • Start the web-server

     yarn dev

7. Access the Application: Once the project is running, access the application through your web browser at http://localhost:3333.
   Additionally:

   • The analytics server is available at http://localhost:9696.
   • The sync server can be accessed at http://localhost:9697.