Developer Guide
Note
This document complements the guidelines set out in the SKA telescope developer portal
Tooling Pre-requisites
This project requires Node and YARN to install and run. To install please follow the instructions for your operating system at nodejs downloads.
Alternatively, the official Node docker image can be used. Instructions can be found on the official Node docker image site.
Development setup
To run the application directly on your host machine
All the following notes assume you are at the command prompt for your chosen environment.
Confirm Node and YARN are installed and configured correctly, both the following commands should return the relevant version number.
> node --version
> yarn --version
Clone the repository and its submodules:
git clone --recursive git@gitlab.com:ska-telescope/sdp/ska-sdp-dataproduct-dashboard.git
make make
Scripts for running, testing, and building the application are defined in the scripts section of the package.json file. These are run using YARN
To run the application locally on your host machine, install all the latest SKAO React components library and other necessary project dependencies with the YARN package manager:
> yarn skao:update
> yarn
Using a .env File (For hosting the application on a developer machine)
Note
This file is excluded from version control using .gitignore to prevent exposing sensitive information.
Create a file named .env at the root of your project directory. The following list of environment variables can be set according to your environment:
SKIP_PREFLIGHT_CHECK='true'
REACT_APP_SKA_SDP_DATAPRODUCT_API_URL='http://localhost:8000'
REACT_APP_API_REFRESH_RATE='1000'
REACT_APP_USE_LOCAL_DATA='false'
REACT_APP_ALLOW_MOCK_AUTH='true'
REACT_APP_MSENTRA_CLIENT_ID='The applciation ID'
REACT_APP_MSENTRA_TENANT_ID='The tenant ID'
REACT_APP_MSENTRA_REDIRECT_URI='http://localhost:8000'
REACT_APP_PERMISSIONS_API_URI='http://localhost:8000/v1/getuserrole'
Running scripts
You should now be able to run the scripts defined in the package.json within the project directory.
Running the application in development mode
The app can be run with the node environment set to NODE_ENV=test (allowing Istanbul to instrument the code) and webpack serve with –mode development. You can access your application at http://localhost:8100. The app will recompile and restart if you make any edits to the source files.
> yarn start
Running the application tests using Cypress
Cypress has been set up to provide component and end to end testing. For information on the use of Cypress, see Cypress component-testing.
Code coverage is implemented with Istanbul and NYC for instrumenting the code, and cobertura reporter as it is used for reporting for the Gitlab CI of coverage statistics.
Cypress can be opened in a browser by running:
> yarn cypress:open
Or alternatively unit and end to end tests can be run headless by:
> yarn test:component:headless
> yarn test:e2e:headless
Code coverage can be viewed by opening the build/coverage/index.html in a browser after running:
> yarn test:coverage:report
Running the production code
The build script builds the app for production to the dist folder. The build is minified and any JSX is transpiled to JavaScript. Your app is ready to be deployed!
> yarn build
Running the application inside a docker container on your host machine
When running the application within a container, the production image of the application is first built in the docker file and a Nginx image is then used to run the application. The following docker commands can be used to build and run it locally:
docker build -t ska-sdp-dataproduct-dashboard .
docker run -p 80:80 ska-sdp-dataproduct-dashboard
The project will then be accessible at the URL http://localhost/
Steps to run the system locally in Minikube
The following steps will assume that you have the repo checked out, or have the chart locally.
Start Minikube if it is not already running:
minikube start
minikube status
If needed, build the Docker images, tag and load them to Minikube.
docker build -t ska-sdp-dataproduct-dashboard .
docker images
docker tag [Image ID] ska-sdp-dataproduct-dashboard:[Tag]
minikube image load ska-sdp-dataproduct-dashboard:[Tag]
minikube image ls
If you want to run the API with a local instance of Elasticsearch, you can add the Bitnami repository to your repositories:
helm repo add bitnami https://charts.bitnami.com/bitnami
Pull and load the Elasticsearch images into Minikube if required:
docker image pull bitnami/elasticsearch:[Tag]
minikube image load bitnami/elasticsearch:[Tag]
Update the DPD chart (Chart.yaml) dependency to match the Elasticsearch tag and enable it.
dependencies:
- name: 'elasticsearch'
version: '[Tag]'
repository: 'https://charts.bitnami.com/bitnami'
condition: elasticsearch.enabled
Change to the chart directory in the repository:
cd charts/ska-sdp-dataproduct-dashboard/. Make the needed changes to image versions and enable the deployments as required in the values files. Then update the Helm dependencies.
helm dependency update .
helm dependency build
Create a new namespace (optional):
kubectl create namespace [namespace]Install the helm chart with the following values:
helm install [deploy-name] charts/ska-sdp-dataproduct-dashboard -n [namespace] –values values_local_deployment.yaml
On a system with limited resources / slow connection, run with the following additional flags:
helm install [deploy-name] charts/ska-sdp-dataproduct-dashboard -n [namespace] --values values_local_deployment.yaml --set diagnosticMode.enabled=true --timeout=60m
Once the above is complete you will have the following running:
The Data Product API
The Data Product Dashboard
To be able to access the API and the dashboard (Add Elasticsearch if in use as well so that it can be reached by the API on the local host) run the following:
kubectl -n [namespace] port-forward service/ska-sdp-dataproduct-api 8000:8000
kubectl -n [namespace] port-forward service/ska-sdp-dataproduct-dashboard 80:80
You should now be able to access the API and the Dashboard on the following URL’s:
To get data onto the PV:
kubectl get pod -n [namespace]
kubectl cp [host path]/ska-sdp-dataproduct-api/tests/test_files/product [ska-sdp-dataproduct-api pod]:/usr/data -n [namespace]