crosspm2

CrossPM Config

import

If defined, imports yaml config parts from other files. Must be the first parameter in config file.

import:
- cred.yaml
- template-config.yaml

Read more about import hacks (RU)

cpm

Main configuration such as manifest file name and cache path.

cpm:
  # Short description of your configuration file
  description: Simple example configuration
  
  dependencies: dependencies.txt
  dependencies-lock: dependencies.txt.lock
  lock-on-success: true
  prefer-local: True 

cpm:dependencies and cpm:dependencies-lock

cpm:lock-on-success

If set to true (or yes or 1) - dependencies lock file will be generated after successfull download. Lock file will be saved near original dependencies file.

cpm: prefer-local

Search in local cache before query repo. Work only with:

cache

Parameters for cache handling

cache:
  cmdline: cache
  env: CROSSPM_CACHE_ROOT
#  default:
#  path:
  storage:
    packed: '{package}/{branch}/{version}/{compiler}/{arch}/{osname}/{package}.{version}.tar.gz'
    unpacked: '{package}/{branch}/{version}/{compiler}/{arch}/{osname}'

Several way to set where crosspm stored files (archive and unpacked package)

cache:storage

Local storage setting, how crosspm will be stored files:

cache:clear - NOT used

cache:
  ...
  # Parameters for cleaning cache.
  clear:
    #  Delete files or folders older than "days".
    days: 10
    # Delete older files and folders if cache size is bigger than "size". Could be in b, Kb, Mb, Gb. Bytes (b) is a default.
    size: 300 mb
    # Call cache check and clear before download
    auto: true

columns

columns: "*package, version, branch"

Let’s keep in mind that any value we use in path, properties and columns description, called column in CrossPM.

Manifest file columns definition. Asterisk here points to name column (column of manifest file with package name). CrossPM uses it for building list with unique packages (i.e. by package name)

Некоторые колонки в crosspm имеют специальное значение

parser column

Чтобы явно задать какой пакет каким парсером искать - используйте parser-колонку в dependencies-файле.

Этот подход позволяет ускорить поиск, если у вас много parser.

config.yaml content:

# Часть пропущена
columns: "*package, version, parser"
parsers:
  release_packages:
    columns:
      version: '{int}.{int}.{int}'
    path: '{server}/{repo}/{name}/{name}.{version}.[msi|exe]'
  feature_packages:
    columns:
      version: '{int}.{int}.{int}'
    path: '{server}/{repo}/{name}/{name}.{version}.[msi|exe]'

dependencies.txt content:

# package version parser
package1 1.0.* release_packages
package2 1.0.* release_packages

# Указывайте несколько через запятую, без пробела:
package2 1.0.* feature_packages,release_packages

# Варианты для поиска по всем парсерам: *, -, пустая колонка
package3 1.0.* -
package3 1.0.* *
package3 1.0.*

repo column

Можно указать в каких репозиториях искать пакет - используйте колонку repo. Все используемые repo должны быть явно заданы в sources.

Указание repo не ускоряет поиск, т.к. запрос идет по определенным в sources репозиториям. Но позволяет точнее определить пакет.

config.yaml content:

columns: "*package, version, repo"
sources:
  - repo: repo.snapshot
    parser: artifactory-parser
  - repo: repo.release
    parser: artifactory-parser

dependencies.txt content:

# package version repo
package1 1.0.* repo.snapshot
package1 1.0.* repo.release

# Указывайте несколько через запятую, без пробела:
package1 1.0.* repo.snapshot,repo.release

# Поиск во всех репозиториях:
package1 1.0.* *
package1 1.0.* -

values

Lists or dicts of available values for some columns (if we need it).

values:
  quality:
    1: banned
    2: snapshot
    3: integration
    4: stable
    5: release

options

Here we can define commandline options and environment variable names from which we will get some of columns values. We can define default values for those columns here too. Each option must be configured with this parameters:

parsers

Rules for parsing columns, paths, properties, etc.

parsers:
  common:
    columns:
      version: "{int}.{int}.{int}[.{int}][-{str}]"
    sort:
      - version
      - '*'
    index: -1
    usedby:
      AQL:
        "@dd.{package}.version": "{version}"
        "@dd.{package}.operator": "="

      property-parser:
        "deb.name": "package"
        "deb.version": "version"
        "qaverdict": "qaverdict"

  artifactory:
    path: "{server}/{repo}/{package}/{branch}/{version}/{compiler|any}/{arch|any}/{osname}/{package}.{version}[.zip|.tar.gz|.nupkg]"
    properties: "some.org.quality = {quality}"

parsers:path

Path template for searching packages in repository. Here {} is column, [|] is variation.

path: "{server}/{repo}/{package}/{compiler|any}/{osname}/{package}.{version}[.zip|.tar.gz]

these paths will be searched:

https://repo.some.org/artifactory/libs-release.snapshot/boost/gcc4/linux/boost.1.60.204.zip
https://repo.some.org/artifactory/libs-release.snapshot/boost/gcc4/linux/boost.1.60.204.tar.gz
https://repo.some.org/artifactory/libs-release.snapshot/boost/any/linux/boost.1.60.204.zip
https://repo.some.org/artifactory/libs-release.snapshot/boost/any/linux/boost.1.60.204.tar.gz

Для того, чтобы задать любой путь, можно использовать *. Это пригодится, когда вам нужно выкачать пакеты, в оригинале не обращающие внимания на пути (nupkg, deb)

# deb
parsers:
  common:
    columns:
      version: '{int}.{int}.{int}[.{int}]'
  artifactory-deb:
    path: '{server}/{repo}/pool/*/{package}.{version}.deb'

# nupkg
parsers:
  common:
    columns:
      version: '{int}.{int}.{int}[.{int}][-{str}]'
  artifactory-nupkg:
    path: '{server}/{repo}/pool/*/{package}.{version}.nupkg'

defaults

Default values for columns not defined in “options”.

defaults:
  branch: master
  quality: stable
  # default support python format, like this:
  otherparams: "{package}/{version}"

solid:ext

Set of rules pointing to packages which doesn’t need to be unpacked. File name extension (i.e. “.tgz”, “.tar.gz”, or more real example “.deb”).

solid:
  ext: *.deb

fails:unique

Here we can define some rules for failing CrossPM jobs. fails:unique - List of columns for generating unique index.

fails:
  unique:
  - package
  - version

common

Common parameters for all or several of sources.

common:
  server: https://repo.some.org/artifactory
  parser: artifactory
  type: jfrog-artifactory
  auth_type: simple
  auth:
  - username
  - password

sources

Sources definition. Here we define parameters for repositories access.

sources:
    - repo:
      - libs-release.snapshot
      - libs-release/extlibs
    - type: jfrog-artifactory
      parser: artifactory
      server: https://repo.some.org/artifactory
      repo: project.snapshot/temp-packages
      auth_type: simple
      auth:
      - username2
      - password2

output:tree

Report output format definition. tree - columns and widths for tree output, printed in the end of CrossPM job.

output:
  tree:
  - package: 25
  - version: 0

Full example

We’ll add some more examples soon. Here is one of configuration file examples for now.

crosspm.yaml

import:
- cred.yaml

cpm:
  description: Simple example configuration
  dependencies: dependencies.txt
  dependencies-lock: dependencies.txt.lock
  lock-on-success: true
  
  # search in local cache before query repo. 
  # Work only with:
  # - FIXED package version, without mask
  # - FIXED extenstion
  # - download-mode only (in lock it does not work)
  # - only in artifactory-aql adapters
  prefer-local: True 
  
cache:
  cmdline: cache
  env: CROSSPM_CACHE_ROOT
  storage:
    packed: '{package}/{branch}/{version}/{compiler}/{arch}/{osname}/{package}.{version}.tar.gz'
    unpacked: '{package}/{branch}/{version}/{compiler}/{arch}/{osname}'

columns: "*package, version, branch"

values:
  quality:
    1: banned
    2: snapshot
    3: integration
    4: stable
    5: release

options:
  compiler:
    cmdline: cl
    env: CROSSPM_COMPILER
    default: vc110

  arch:
    cmdline: arch
    env: CROSSPM_ARCH
    default: x86

  osname:
    cmdline: os
    env: CROSSPM_OS
    default: win

parsers:
  common:
    columns:
      version: "{int}.{int}.{int}[.{int}][-{str}]"
    sort:
      - version
      - '*'
    index: -1
    usedby:
      AQL:
        "@dd.{package}.version": "{version}"
        "@dd.{package}.operator": "="

      property-parser:
        "deb.name": "package"
        "deb.version": "version"
        "qaverdict": "qaverdict"

  artifactory:
    path: "{server}/{repo}/{package}/{branch}/{version}/{compiler|any}/{arch|any}/{osname}/{package}.{version}[.zip|.tar.gz|.nupkg]"
    properties: "some.org.quality = {quality}"
    
defaults:
  branch: master
  quality: stable
  # default support python format, like this:
  otherparams: "{package}/{version}"

solid:
  ext: *.deb

fails:
  unique:
  - package
  - version

common:
  server: https://repo.some.org/artifactory
  parser: artifactory
  type: jfrog-artifactory
  auth_type: simple
  auth:
  - username
  - password

sources:
- repo:
  - libs-release.snapshot
  - libs-release/extlibs

- type: jfrog-artifactory
  parser: artifactory
  server: https://repo.some.org/artifactory
  repo: project.snapshot/temp-packages
  auth_type: simple
  auth:
  - username2
  - password2

output:
  tree:
  - package: 25
  - version: 0