Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
G
geomag-algorithms
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
ghsc
National Geomagnetism Program
geomag-algorithms
Commits
9ef83b37
Commit
9ef83b37
authored
2 years ago
by
Jeremy M Fee
Browse files
Options
Downloads
Patches
Plain Diff
Update pipeline to use code.usgs.gov images
parent
bbb4d026
No related branches found
No related tags found
1 merge request
!188
Pipeline updates
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
.gitignore
+6
-5
6 additions, 5 deletions
.gitignore
.gitlab-ci.yml
+34
-20
34 additions, 20 deletions
.gitlab-ci.yml
pyproject.toml
+3
-0
3 additions, 0 deletions
pyproject.toml
with
43 additions
and
25 deletions
.gitignore
+
6
−
5
View file @
9ef83b37
__pycache__
.coverage
cov.xml
.DS_Store
.eggs
*.pyc
coverage.xml
.ipynb_checkpoints*
.mypy_cache
.pytest_cache
htmlcov
.vscode
*.egg-info
*.pyc
build
coverage.xml
dist
*.egg-info
\ No newline at end of file
htmlcov
junit.xml
This diff is collapsed.
Click to expand it.
.gitlab-ci.yml
+
34
−
20
View file @
9ef83b37
image
:
${DEVOPS_REGISTRY}
usgs/python:3.
9
-obspy
image
:
code.usgs.gov:5001/devops/images/
usgs/python:3.
10
-obspy
stages
:
-
init
...
...
@@ -28,7 +28,6 @@ workflow:
-
IMAGE_NAME=usgs/${APP_NAME}:${CI_COMMIT_REF_SLUG}
-
IMAGE_NAME=${IMAGE_NAME/:master/:latest}
-
INTERNAL_IMAGE_NAME=${CODE_REGISTRY_IMAGE}/${IMAGE_NAME}
-
STACK_NAME=${APP_NAME}
.deploy
:
extends
:
...
...
@@ -90,6 +89,22 @@ workflow:
APP_DEPLOY_DIR
:
"
/geomag/geomag-algorithms"
REQUIRED_PREFIX
:
"
/geomag"
# template for jobs that need docker-in-docker
.dind
:
# TODO: refactor Docker build to different runner
# before_script:
# - |
# echo "${CI_REGISTRY_PASSWORD}" | docker login \
# --username "${CI_REGISTRY_USER}" \
# --password-stdin \
# "${CI_REGISTRY}"
image
:
code.usgs.gov:5001/devops/images/usgs/docker:20
services
:
-
alias
:
docker
name
:
code.usgs.gov:5001/devops/images/usgs/docker:20-dind
variables
:
DOCKER_DRIVER
:
overlay2
# rules to define which branches should trigger actions
.development-env
:
&development-env
if
:
$CI_PROJECT_PATH != $UPSTREAM_PATH
...
...
@@ -106,7 +121,7 @@ workflow:
.production-env
:
&production-env
if
:
>
$CI_PROJECT_PATH == $UPSTREAM_PATH
&& ( $CI_COMMIT_BRANCH == 'production' || $CI_COMMIT_TAG)
&& ( $CI_COMMIT_BRANCH == 'production' || $CI_COMMIT_TAG
)
variables
:
ENVIRONMENT
:
production
...
...
@@ -127,7 +142,6 @@ Poetry:
# install into .venv for artifact
-
poetry config virtualenvs.in-project
true
--local
-
poetry install
-
poetry run safety check
stage
:
init
variables
:
PIP_CACHE_DIR
:
"
$CI_PROJECT_DIR/.cache/pip"
...
...
@@ -136,21 +150,28 @@ Poetry:
# Test Stage
## --------------------------------------------------
Python Build
:
Audit
:
needs
:
-
Poetry
script
:
-
poetry run poe audit
stage
:
test
Build
:
needs
:
-
Poetry
script
:
-
poetry build
stage
:
test
Python
Lint
:
Lint
:
needs
:
-
Poetry
script
:
-
poetry run
black --check .
-
poetry run
poe lint
stage
:
test
Python
Test
:
Test
:
artifacts
:
reports
:
coverage_report
:
...
...
@@ -160,8 +181,7 @@ Python Test:
needs
:
-
Poetry
script
:
-
poetry run pytest --cov=geomagio --junitxml junit.xml
-
poetry run coverage xml
-
poetry run poe test
stage
:
test
## --------------------------------------------------
...
...
@@ -171,9 +191,9 @@ Python Test:
Build Docker Image
:
extends
:
-
.adjust_image_names
image
:
${DEVOPS_REGISTRY}docker:19.03-git
-
.dind
needs
:
-
Python
Build
-
Build
script
:
-
LOCAL_IMAGE="local/${IMAGE_NAME}"
## build image
...
...
@@ -186,8 +206,6 @@ Build Docker Image:
"."
## trivy scan before push
-
wget https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz
-
tar zxvf trivy_${TRIVY_VERSION}_Linux-64bit.tar.gz
# fail LOW,MEDIUM vulnerabilities that have a fix available
-
./trivy image --exit-code 1 --ignore-unfixed --severity LOW,MEDIUM "${LOCAL_IMAGE}";
# fail HIGH,CRITICAL vulnerabilities
...
...
@@ -203,17 +221,13 @@ Build Docker Image:
docker tag "${LOCAL_IMAGE}" "${IMAGE}";
docker push "${IMAGE}";
done
services
:
-
alias
:
docker
name
:
${DEVOPS_REGISTRY}docker:19.03-dind
stage
:
integration
tags
:
# TODO: refactor to separate build/publish steps
-
build
variables
:
APP_NAME
:
geomag-algorithms
DOCKER_DRIVER
:
overlay2
FROM_IMAGE
:
${DEVOPS_REGISTRY}usgs/python:3.9-obspy
TRIVY_VERSION
:
"
0.27.1"
FROM_IMAGE
:
code.usgs.gov:5001/devops/images/usgs/python:3.10-obspy
## --------------------------------------------------
# Deploy Stage
...
...
This diff is collapsed.
Click to expand it.
pyproject.toml
+
3
−
0
View file @
9ef83b37
...
...
@@ -78,6 +78,9 @@ copy-absolutes = "geomagio.processing.copy_absolutes:main"
[tool.poe.tasks]
# e.g. "poetry run poe lint"
audit
=
[
{
shell
=
"safety check"
}
]
lint
=
[
{
shell
=
"black --check ."
}
,
# TODO: fix isort warnings then enable this check
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment