Package installations and upgrades on a *nix server

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

System patches, software upgrades, and software installations are risky processes in a production environment. They can lead to an outage or incident if the software installation introduces compatibility issues with critical processes and applications running on the host. You want a search that will show you recent software changes on a host, so you can more easily identify what changed and when while investigating an incident.

Data required

*nix: Operating system logs

Procedure

Run the following search. You can optimize it by specifying an index and adjusting the time range.

((tag=update) OR (yum ("Updated:" OR "Installed:")) OR (sourcetype=dnf ("Installed:" OR "Upgraded:"))) (NOT Downloading:) host=*
|rex "(?s)(Upgraded|Updated):(?<pkgs_updated>.*?(?=Installed:|\Z))"
|rex "(?s)Installed:(?<pkgs_installed>.*)"
|rex mode=sed field=pkgs_updated "s/\s+/ /g"
|rex mode=sed field=pkgs_installed "s/\s+/ /g"
|eval pkgs_updated=split(pkgs_updated, " "), pkgs_installed=split(pkgs_installed, " ")
|table _time host pkgs* _raw

Search explanation

The table provides an explanation of what each part of this search achieves. You can adjust this query based on the specifics of your environment.

Splunk Search	Explanation
`((tag=update) OR (yum ("Updated:" OR "Installed:")) OR (sourcetype=dnf ("Installed:" OR "Upgraded:"))) (NOT Downloading:)`	Return all software installations and upgrades in your environment.
`host=*`	Search on all hosts.
`\|rex "(?s)(Upgraded\|Updated):(?<pkgs_updated>.*?(?=Installed:\|\Z))"`	Capture everything between `Upgraded: OR Updated`: and `=Installed` or the end of the string.
`\|rex "(?s)Installed:(?<pkgs_installed>.*)"`	Capture everything after Installed:.
`\|rex mode=sed field=pkgs_updated "s/\s+/ /g"`	Replace any white space with a single space in the string pointed to by `pkgs_updated`.
`\|rex mode=sed field=pkgs_installed "s/\s+/ /g"`	Replace any white space with a single space in the string pointed to by `pkgs_installed`.
`\|eval pkgs_updated=split(pkgs_updated, " "), pkgs_installed=split(pkgs_installed, " ")`	Split the listed field by the single space added in the rex commands.
`\|table _time host pkgs* _raw`	Display the results in a table with columns in the order shown.

Next steps

Sample results for this search are shown in the table below. It shows the time of the update or upgrade, the affected host, and the name of the package that has either installed or been updated.

`_time`	`host`	`pkgs_installed`	`pkgs_updated`
`2020-09-03T12:37:16.000-0700`	`ip-172-31-64-114`	`httpd-tools-2.4.43-1.amzn2.x86_64`
`2020-09-03T12:37:15.000-0700`	`ip-172-31-64-114`	`apr-util-bdb-1.6.1-5.amzn2.0.2.x86_64`
`2020-09-03T12:37:15.000-0700`	`ip-172-31-64-114`	`apr-1.6.3-5.amzn2.0.2.x86_64`
`2020-09-03T08:50:02.000-0700`	`ip-172-31-27-100.us-west-1.compute.internal`	`grub2-tools-efi-1:2.02-87.el8_2.x86_64` `kernel-4.18.0-193.14.3.el8_2.x86_64` `kernel-core-4.18.0-193.14.3.el8_2.x86_64` `kernel-modules-4.18.0-193.14.3.el8_2.x86_64` `linux-firmware-20191202-97.gite8a0f4c9.el8.noarch`	`NetworkManager-1:1.22.8-5.el8_2.x86_64` `NetworkManager-libnm-1:1.22.8-5.el8_2.x86_64` `NetworkManager-team-1:1.22.8-5.el8_2.x86_64` `NetworkManager-tui-1:1.22.8-5.el8_2.x86_64` `bind-export-libs-32:9.11.13-5.el8_2.x86_64` `ca-certificates-2020.2.41-80.0.el8_2.noarch` `cloud-init-19.4-1.el8.7.noarch` `coreutils-8.30-7.el8_2.1.x86_64` `coreutils-common-8.30-7.el8_2.1.x86_64` `dbus-1:1.12.8-10.el8_2.x86_64` `dbus-common-1:1.12.8-10.el8_2.noarch` `dbus-daemon-1:1.12.8-10.el8_2.x86_64` `dbus-libs-1:1.12.8-10.el8_2.x86_64` `dbus-tools-1:1.12.8-10.el8_2.x86_64` `dnf-4.2.17-7.el8_2.noarch` `dnf-data-4.2.17-7.el8_2`

Finally, you might be interested in other processes associated with the Maintaining *nix systems use case.