External repositories
Infrahub supports two different types of connections to external Git repositories:
- Repository fully integrates with Git version control, including branch tracking and two-way branch synchronization.
- Read-only Repository links a particular branch in Infrahub to a particular ref in the Git repository. It will only read from the Git repository. It will never make any changes to the external repository.
Recommendations
- We recommend to use a dedicated repository for Infrahub resources.
- We recommend to configure the branch
main
asdefault_branch
. - Use
Read-only Repository
preferably for slow moving assets as pulling the changes is a manual operation. It goes well with putting a release tag as a ref and bump the ref from release to release. - Use
Read-only Repository
for shared assets across multiple instances or repository external to your organization.
Adding a repository
You will need to submit an access token with your request to create a repository in Infrahub. Infrahub uses your username and this token to connect to the external Git repository.
If your repository is private you might want to have a personal token to access the repository. If you use GitHub as your Git Server you can find further information in this Guide
Via the web interface
- Log in to the Infrahub UI
- Go to
Unified Storage
>Repository
- Click on the
+ Add Git Repository
button - Select the repository type
- Complete the required information:
- Repository location
- Authentication
- Name
- Description
- Ref
- Tags
The URL of the external repository, for example https://github.com/opsmill/infrahub.git
.
(Optional): Credential object holding your username / password or personal token with access to the Git repository specified.
The name you want to give the repository in Infrahub for identification purposes.
(Optional): A description or comment about the repository used for informational purposes.
(Optional): Branch, tag or commit reference to pull.
(Optional): Assign any tags to be associated with the repository.
In the UI, you should see your new repository. If the repository you added has Unknown
as Operational Status it means that Infrahub didn't managed to reach your repository. Verify the location and credentials.
Via the GraphQL interface
Using the GraphQL Interface, it is possible to add a Repository
or Read-only Repository
via a Mutation.
- Open the GraphQL Interface.
- If needed, i.e., your repository is private, create a Credential object to hold your username / password.
# Endpoint: http://127.0.0.1:8000/graphql/main
mutation {
CorePasswordCredentialCreate(
data: {
name: {value: "my-git-credential"},
username: {value: "MY_USERNAME"},
password: {value: "MY_TOKEN_OR_PASSWORD"}
}
) {
ok
object {
hfid
}
}
}
- Copy-paste the correct mutation from below and complete the information
- Repository
- Read-only Repository
# Endpoint: http://127.0.0.1:8000/graphql/main
mutation {
CoreRepositoryCreate(
data: {
name: {value: "My Git Repository"},
location: {value: "https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git"},
# The HFID returned during step 2 will be the name of the credentials
credential: {hfid: "my-git-credential"}
}
) {
ok
object {
id
}
}
}
Make sure that you are on the correct Infrahub branch. Unlike a Repository, a Read-only Repository will only pull files into the Infrahub branch on which it was created.
# Endpoint : http://127.0.0.1:8000/graphql/<branch>
mutation {
CoreReadOnlyRepositoryCreate(
data: {
name: {value: "My Git Repository"},
location: {value: "https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git"},
ref: { value: "BRANCH/TAG/COMMIT_TO_TRACK" },
# Optional: the HFID returned during step 2 will be the name of the credentials
credential: {hfid: "my-git-credential"}
}
) {
ok
object {
id
}
}
}
The repository should be visible under Unified Storage / Repository. If the repository you added has Unknown
as Operational Status it means that Infrahub didn't managed to reach your repository. Verify the location and credentials.
Via the Infrahub SDK
- Install and setup the Infrahub SDK
- If needed, i.e., your repository is private, create a Credential object to hold your username / password. We highly recommend to provide credentials for Repository to allow branch creation from Infrahub.
# Create credential object ...
credential = client.create(
"CorePasswordCredential",
name="My Git Credential",
username="MY_USERNAME",
password="MY_TOKEN_OR_PASSWORD",
)
# ... and save it!
credential.save()
- Create the correct repository object
- Repository
- Read-only Repository
# Create repository object ...
repository = client.create(
"CoreRepository",
name="My Git repository",
location="https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git",
# The credential object created at step 2
credential=credential,
)
# and save it ...
repository.save()
# Create repository object ...
repository = client.create(
"CoreReadOnlyRepository",
name="My Git repository",
location="https://GIT_SERVER/YOUR_GIT_USERNAME/YOUR_REPOSITORY_NAME.git",
ref="BRANCH/TAG/COMMIT_TO_TRACK",
# Optional: the credential object created at step 2
credential=credential,
)
# and save it ...
repository.save()
Pulling changes for read-only repository
Read-only Repository
and Repository
work in different ways when it comes to tracking changes on the remote repository. Please refer to the Repository Topic for further details.
Unlike Repository
, Infrahub does not automatically update Read-only Repository
with changes from the external repository. To pull in changes from the external repository you must update the ref
of the Read-only Repository
to the desired value.
Via the web interface
- Log in to the Infrahub UI
- Go to
Unified Storage
>Repository
- Click on the
CoreReadOnlyRepository
record - Click on the
Edit Read-Only Repository
button - Change the required information:
- Ref
(Optional): Branch, tag or commit reference to pull.
Via the GraphQL interface
- Open the GraphQL Interface.
- Copy-paste the correct mutation from below and complete the information
# Endpoint : http://127.0.0.1:8000/graphql/main
mutation {
CoreReadOnlyRepositoryUpdate(
data: {
hfid: ["My Git repository"]
ref: { value: "BRANCH/TAG/COMMIT_TO_TRACK" }
}
) {
ok
object {
id
}
}
}
Troubleshooting repository
Various issues could affect repositories. The Repository Status gives you further details about the root cause.
Also, directly from the UI you can access the output of low level repository operations:
- Log in to the Infrahub UI
- Go to
Unified Storage
>Repository
- Click on the
CoreReadOnlyRepository
record - Click on the
Tasks
tab
GitHub access token
If you are using GitHub as your Git Server and your repository is private, you need to have an access token to be able to access the repository. You can use either a classic Personal Access Token or a Fine-grained Access Token.
Generate a GitHub fine-grained personal access token
- Go to settings > Developer Settings > Personal access tokens New GitHub token
- Select Fine-grained tokens
- Limit the scope of the token in Repository Access > Only Select Repositories
- Grant the token permission:
- a. If you want to create a
Repository
using this token, then you will need to give itRead/Write
access for the Content of the repository. - b. If you want to create a
Read-only Repositor
y using this token, then you will only need to give itRead
access for the Content of the repository.
Installing custom CA certificates
If the git server hosting your external git repository is using a server certificate that is signed by a certificate authority that is not part of trust store, then you will have to add the CA certificate using the following process.
In this process we will have to build a custom docker image. This docker image will have to be rebuild every time you want to use a new release.
First you will have to gather the CA certificate (and any intermediate CA certificates) in PEM format and store it on you local disk.
This guide makes the assumption that you have saved the CA certificate file as mycacertificate.crt
Next we will have to create a Dockerfile. The file should be saved as Dockerfile
in the same directory as the directory containing the CA certificate you want to install.
ARG INFRAHUB_VERSION=latest
FROM registry.opsmill.io/opsmill/infrahub:${INFRAHUB_VERSION}
COPY mycacertificate.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates
We then have to build a Docker image from the Dockerfile. We can do this by executing this command
INFRAHUB_VERSION=latest && docker build --build-arg INFRAHUB_VERSION=$INFRAHUB_VERSION -f Dockerfile -t custom/infrahub:${INFRAHUB_VERSION} .
This will build a custom docker image and tag it as custom/infrahub:latest
. You can change the version by changing the version we set in the INFRAHUB_VERSION
shell variable and you can change the tag to you own preference.
As a last step we have to create a docker-compose.override.yml
file with the following contents in the development
directory of your clone of the Infrahub repository.
---
services:
task-worker:
image: custom/infrahub:latest
A development environment can then be spun up with invoke demo.start
command as, explained in the Installing Infrahub guide.
Disable certificate verification
You can disable certificate verification for the external Git repository by following the following process.
Disabling certificate validation is a BAD practice and is STRONGLY discouraged
In this process we will have to build a custom docker image. This docker image will have to be rebuild every time you want to use a new release.
We will have to create a Dockerfile, that should be saved as Dockerfile
on your local filesystem.
ARG INFRAHUB_VERSION=latest
FROM registry.opsmill.io/opsmill/infrahub:${INFRAHUB_VERSION}
RUN git config --global http.sslVerify "false"
We then have to build a Docker image from the Dockerfile. We can do this by executing this command
INFRAHUB_VERSION=latest && docker build --build-arg INFRAHUB_VERSION=$INFRAHUB_VERSION -f Dockerfile -t custom/infrahub:${INFRAHUB_VERSION} .
This will build a custom docker image and tag it as custom/infrahub:latest
. You can change the version by changing the version we set in the INFRAHUB_VERSION
shell variable and you can change the tag to you own preference.
As a last step we have to create a docker-compose.override.yml
file with the following contents in the development
directory of your clone of the Infrahub repository.
---
services:
task-worker:
image: custom/infrahub:latest
A development environment can then be spun up with invoke demo.start
command as, explained in the Installing Infrahub guide.
Using a proxy server
In some scenarios Infrahub's git worker containers are not allowed to connect directly to the git server hosting your repository and the connection has to pass through a proxy server.
This way of using a proxy server for git repositories only works when using HTTP(S) connections. SSH connections are not supported.
In this process we will have to build a custom docker image. This docker image will have to be rebuild every time you want to use a new release.
We will have to create a Dockerfile, that should be saved as Dockerfile
on your local filesystem. In the Dockerfile you will have to adapt the URL for the proxy to your environment. Replace user:password
with the username and password you have to use for the proxy, if required. Replace the FQDN internal.proxy
to the FQDN of the proxy and modify the port, if required.
ARG INFRAHUB_VERSION=latest
FROM registry.opsmill.io/opsmill/infrahub:${INFRAHUB_VERSION}
RUN git config --global http.proxy http://user:password@internal.proxy:8080
We then have to build a Docker image from the Dockerfile. We can do this by executing this command
INFRAHUB_VERSION=latest && docker build --build-arg INFRAHUB_VERSION=$INFRAHUB_VERSION -f Dockerfile -t custom/infrahub:${INFRAHUB_VERSION} .
This will build a custom docker image and tag it as custom/infrahub:latest
. You can change the version by changing the version we set in the INFRAHUB_VERSION
shell variable and you can change the tag to you own preference.
As a last step we have to create a docker-compose.override.yml
file with the following contents in the development
directory of your clone of the Infrahub repository.
---
services:
task-worker:
image: custom/infrahub:latest
A development environment can then be spun up with invoke demo.start
command as, explained in the Installing Infrahub guide.