Skip to content

Architecture of the Pub/Sub Integration

Preview

This feature is not subject to our service terms. Pre-GA features are available as is and might have limited support.

This document describes the architecture of the Modigie Integration for Cloud Pub/Sub API and the infrastructure involved.

Modigie Pub/Sub Repository

Your application connects to a Modigie Pub/Sub Repository that is exclusive to your application. A repository represents an instance of the Modigie Integration for Cloud Pub/Sub API and is owned by your Modigie Org Account. The repository is associated with an active subscription, which is charged for completed jobs. The repository is used to configure the service's behavior and the infrastructure required for your application.

The repository also functions as a container for access control. All data that your application sends to the service is encapsulated in this repository and isolated from other applications' repositories.

Important

Modigie encrypts all data at rest and in transit.

Connection between your application and your Pub/Sub repository at Modigie

Connection between your application and your Pub/Sub repository at Modigie

Infrastructure overview

The Modigie Integration for Cloud Pub/Sub API uses shared and tenant-specific infrastructure.

Shared infrastructure

The shared infrastructure, used by all repositories, includes:

  • The Modigie Integration for Cloud Pub/Sub service: This service is hosted on Google Cloud Platform (GCP) within Modigie's production environment. It uses shared Infrastructure as Code (IaC) provisioned through CI/CD pipelines following GitOps practices. The production environment is strictly isolated, and access is controlled using the principle of least privilege. Service updates are deployed gradually without maintenance windows.

Important

Although the API preview is hosted in a preview environment instead of the production environment, we follow the same protocols and configurations in this preview environment. However, at the end of the preview, we will destroy all related data.

Tenant-specific Modigie API resources

When you connect your application to Modigie, the following dedicated resources are provisioned for you:

  • Modigie repository: This repository manages the connection's behavior and features. It also isolates your instance's data from other instances.

  • Modigie job pipeline: While your Pub/Sub repository has the status active or inactive, Modigie also provisions a dedicated job pipeline that reserves real-time capacities for your application based on the subscription.

  • Repository subscription: The active subscription determines the available features of your repository and how you are charged for completed jobs.

  • Quota accounts: The quota accounts limit the rate and volume of jobs that your application can process. The specific limits are determined by the subscription.

Tenant-specific GCP resources

Your Pub/Sub repository and its corresponding job pipeline have their own dedicated Google Cloud resources to support the expected functioning. These resources are hosted within the Google Cloud project provisioned exclusively for your Modigie Org Account.

Important

Unless mentioned otherwise, access to such tenant GCP resources is restricted to Modigie systems and operations based on least privilege practice.

Once your repository is available, we will share the names of all Pub/Sub topics and pull subscriptions, and the email address of your application's service account, the Application Account.

In the examples below, we use the following variables:

  • REPO_SHORT_HASH: The first 15 characters of the SHA256 hex-digest hash of your Pub/Sub repository's unique ID.
  • TENANT_PROJECT_ID: The GCP project ID that we have provisioned for your Modigie Org Account. This project hosts all tenant-specific GCP resources that we create for you across all Modigie products and services.

Google Cloud Pub/Sub Topics

Your application publishes job requests into one of the available Cloud Pub/Sub topics. For each job type, there is a dedicated topic that also performs schema validation on the messages your application publishes. They are available based on the subscription and configuration. The naming conventions are as follows:

  • Enrich Mobile & Verify Employment: inpubsub-job-request-enrichmobile-{REPO_SHORT_HASH}
  • Verify Mobile: inpubsub-job-request-verifymobile-{REPO_SHORT_HASH}
  • Verify Employment (Hygiene): inpubsub-job-request-verifyemploymenthygiene-{REPO_SHORT_HASH}

For your application to publish messages to these topics, it needs to authenticate with the Application Account mentioned below.

Important

Any access to the topics is restricted to your Application Account and Modigie.

Google Cloud Pub/Sub Subscriptions

Your application will receive job response messages through a job type specific Cloud Pub/Sub subscription.

If your application is configured to actively pull job response messages, a Cloud Pub/Sub pull subscription is created for each available job type. For your application to pull messages, it needs to authenticate with the Application Account mentioned below.

  • Enrich Mobile & Verify Employment: inpubsub-job-response-enrichmobile-pull-all-externalendpoint-{REPO_SHORT_HASH}
  • Verify Mobile: inpubsub-job-response-verifymobile-pull-all-externalendpoint-{REPO_SHORT_HASH}
  • Verify Employment (Hygiene): inpubsub-job-response-verifyemploymenthygiene-pull-all-externalendpoint-{REPO_SHORT_HASH}

Important

Any access to the subscriptions is restricted to your Application Account and Modigie.

Custom Google Service Account

Your application uses this Application Account to authenticate with GCP when publishing messages to the aforementioned topics or pulling job responses from the pull subscriptions. This Application Account is a custom Google Service Account designed for secure service-to-service communication.

If you use push mode instead of pull mode, the pushed messages are signed by this service account, and your application should verify the token.

inpubsub-app-{REPO_SHORT_HASH}@{TENANT_PROJECT_ID}.iam.gserviceaccount.com

Info

As part of the configuration, inform Modigie support which of your email addresses should have the necessary permissions to complete the secure configuration on the side of your application. This email address should be a GCP account.

For more information about the authentication, please refer to Authentication.