AbdelHilmi
Docs

Documenting progress of examining os packages version tracking

Abdelfattah Hilmi •
#Cloud#Devsecops#SRE

Tracking tools:

Repology Api

Endpoint : https://repology.org/api/v1/

How to get data related to a project across distros by default the api returns the first 200 metions of the project searched this example prints the first 10 :

import requests

url = "https://repology.org/api/v1/project/firefox"

response = requests.request("GET", url)
print(response.json()[:10])

The API returns:

[
	{
		"repo": "archlinux32_i486",
		"subrepo": "extra",
		"srcname": "firefox-i18n",
		"binname": "firefox-i18n-is",
		"visiblename": "firefox-i18n-is",
		"version": "108.0.2",
		"licenses": [
			"MPL",
			"GPL",
			"LGPL"
		],
		"summary": "Icelandic language pack for Firefox",
		"status": "outdated",
		"origversion": "108.0.2-1.0"
	},
	{
		"repo": "archlinux32_i486",
		"subrepo": "extra",
		"srcname": "firefox-i18n",
		"binname": "firefox-i18n-zh-cn",
		"visiblename": "firefox-i18n-zh-cn",
		"version": "108.0.2",
		"licenses": [
			"MPL",
			"GPL",
			"LGPL"
		],
		"summary": "Chinese (Simplified) language pack for Firefox",
		"status": "outdated",
		"origversion": "108.0.2-1.0"
	},
	{
		"repo": "arch",
		"subrepo": "extra",
		"srcname": "firefox-i18n",
		"binname": "firefox-i18n-da",
		"visiblename": "firefox-i18n-da",
		"version": "110.0.1",
		"licenses": [
			"MPL",
			"GPL",
			"LGPL"
		],
		"summary": "Danish language pack for Firefox",
		"status": "newest",
		"origversion": "110.0.1-1"
	},

    ...]

How to Filter data using url query parameters

filters are :
search: project name substring to look for
maintainer: return projects maintainer by specified person
category: return projects with specified category
inrepo: return projects present in specified repository
notinrepo: return projects absent in specified repository
repos: return projects present in specified number of repositories (exact values and open/closed ranges are allowed, e.g. 1, 5-, -5, 2-7)
families: return projects present in specified number of repository families (for instance, use 1 to get unique projects)
repos_newest: return projects which are up to date in specified number of repositories
families_newest: return projects which are up to date in specified number of repository families
newest: return newest projects only
outdated: return outdated projects only
problematic: return problematic projects only

Example: get the newest version of every package which has a name with python3 as a substring in the repo ubuntu_22_04 (theoritically, but the API returns all repos of debian based distros)

import requests

url = "https://repology.org/api/v1/projects/?search=python3&inrepo=ubuntu_22_04&newest=1"

response = requests.request("GET", url)
print(response.json())

the API returns:

{
	"python3-antlr4": [
		{
			"repo": "pardus_21",
			"subrepo": "main",
			"srcname": "python3-antlr4",
			"visiblename": "python3-antlr4",
			"version": "4.9.1",
			"maintainers": [
				"team+python@tracker.debian.org",
				"crusoe@debian.org"
			],
			"categories": [
				"misc"
			],
			"status": "unique",
			"origversion": "4.9.1-1"
		},
		{
			"repo": "parrot",
			"subrepo": "parrot-updates/main",
			"srcname": "python3-antlr4",
			"visiblename": "python3-antlr4",
			"version": "4.9.1",
			"maintainers": [
				"team+python@tracker.debian.org",
				"crusoe@debian.org"
			],
			"categories": [
				"misc"
			],
			"status": "unique",
			"origversion": "4.9.1-1"
		},
        ...]

pkgs.org Api

API is not free !!! we need to scrape serch results if possible (they have captcha)

openHub

presents valuable insights about package maintainace, Activity, vulns+sast report, contributors … but does not specify releases/versions

Other tools I’m checking:

← Back to Blog