project/opkg-lede.git
2 years agolibopkg: support https_proxy master
Vladislav Grigoryev [Wed, 1 Sep 2021 05:48:08 +0000 (08:48 +0300)]
libopkg: support https_proxy

Add support for https_proxy since feeds switched to HTTPS.
In general case, https_proxy may not match http_proxy.
Process http_proxy, https_proxy, and ftp_proxy separately.

Signed-off-by: Vladislav Grigoryev <vg.aetera@gmail.com>
2 years agoopkg_remove: avoid remove pkg repeatly with option --force-removal-of-dependent-packages
Hongxu Jia [Tue, 7 Sep 2021 22:42:45 +0000 (18:42 -0400)]
opkg_remove: avoid remove pkg repeatly with option --force-removal-of-dependent-packages

While remove pkg with '--force-removal-of-dependent-packages',
pkg may be added to pkgs remove list multiple times, add status
check to make sure pkg only be removed once.

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Paul Barker <paul@paulbarker.me.uk>
(Cherry picked from https://git.yoctoproject.org/cgit/cgit.cgi/opkg/commit/?id=e8996180833aa35d101fbeadec3d787ce0bbda5c)
(Fixes https://dev.archive.openwrt.org/ticket/18320 and openwrt/packages CI)

2 years agolibopkg: set 'const' attribute for argv
Daniel Golle [Sun, 9 Jan 2022 20:03:44 +0000 (20:03 +0000)]
libopkg: set 'const' attribute for argv

Just not to make it messier than it already is.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2 years agolibopkg: pkg_hash: prefer to-be-installed packages
Daniel Golle [Sun, 9 Jan 2022 19:52:40 +0000 (19:52 +0000)]
libopkg: pkg_hash: prefer to-be-installed packages

Increase PROVIDES candidate score also in case the package name matches
a to-be-installed package listed.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
2 years agolibopkg: pkg_hash: print unresolved dependencies
Hauke Mehrtens [Sun, 2 May 2021 20:46:02 +0000 (22:46 +0200)]
libopkg: pkg_hash: print unresolved dependencies

When a package is not installed because it has unresolved dependencies
normally we get only an error message like this:
 * pkg_hash_fetch_best_installation_candidate: Packages for ltq-vdsl-app found, but incompatible with the architectures configured
 * opkg_install_cmd: Cannot install package ltq-vdsl-app.

Log in addition the following error message:
 * pkg_hash_check_unresolved: cannot find dependency ltq-dsl-base for ltq-vdsl-app

Fixes: FS#3814
Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
3 years agolibopkg: pkg_hash: prefer original packages to satisfy dependencies
Matthias Schiffer [Sat, 13 Mar 2021 01:00:40 +0000 (02:00 +0100)]
libopkg: pkg_hash: prefer original packages to satisfy dependencies

When one package "provides" another non-virtual package, prefer to use
the original package instead of the providing package.

Example:

Consider packages "foo" and "bar", where "foo" provides "bar".
The current code will sort all candidates by name and use the last entry
by default, so "foo" would be used to satisfy a dependency on "bar".
Change the logic to prefer the actual package "bar" in this case.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
3 years agolibopkg: add option to strip ABI versions from listed names
Daniel Golle [Sun, 14 Mar 2021 22:28:05 +0000 (22:28 +0000)]
libopkg: add option to strip ABI versions from listed names

Listing packages without the ABI versions appended to their names is
needed in some situations. Add a new command line option '--strip-abi'
for that which affects the 'list' and 'list-installed' command.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
3 years agopkg: pass-through ABIVersion to status file
Daniel Golle [Wed, 10 Mar 2021 23:39:48 +0000 (23:39 +0000)]
pkg: pass-through ABIVersion to status file

This allows removing it from package names if needed without
storing the SourceName for each and every package, which is mostly
redundant information.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
3 years agopkg_hash: pkg_hash_check_unresolved: fix segfault
Yousong Zhou [Thu, 24 Dec 2020 08:10:17 +0000 (16:10 +0800)]
pkg_hash: pkg_hash_check_unresolved: fix segfault

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
3 years agolibopkg: remove "extra_data" option
Paul Spooren [Thu, 24 Sep 2020 20:54:33 +0000 (10:54 -1000)]
libopkg: remove "extra_data" option

The "extra_data" allowed to add an extra option behind a repository
definition. This was only ever used to either ignore a specific feed by
setting it to "__dummy__" or adding a sub-folder in the repository path,
as done for Debian repositories.

Instead of using "__dummy__" it is cleaner to "comment out" the specific
repository and Debian repository structure is nowhere used in the
OpenWrt ecosystem, therefore it seem save to remove "extra_data".

Signed-off-by: Paul Spooren <mail@aparcar.org>
3 years agolibopkg: remove support for "dist" config
Paul Spooren [Thu, 24 Sep 2020 20:54:32 +0000 (10:54 -1000)]
libopkg: remove support for "dist" config

The "dist" config option is legacy to imitate Debians repository
structure. However this is never used within the OpenWrt ecosystem and
can therefore be removed.

Signed-off-by: Paul Spooren <mail@aparcar.org>
3 years agolibopkg: fix md5sum calculation
Alexander Ryzhov [Sun, 13 Sep 2020 12:42:56 +0000 (14:42 +0200)]
libopkg: fix md5sum calculation

This regression prevents MD5 checksums from being checked. Packages are
still installed, but this raises several issues:

- if only MD5 checksums are provided in the package list, it is trivial
  for an attacker to modify the content of a package, since checksum
  verification is bypassed. If both MD5 and SHA256 checksums are
  provided, then SHA256 is correctly verified and the attack is not
  possible.

- future efforts to harden checksum verification would prevent package
  installation.

Note that OpenWrt has switched to SHA256 for all its packages several
years ago. As a result, this bug does not affect OpenWrt packages from the
official package feeds.

However, custom package repositories that only use MD5 are affected.

Initially submitted at https://github.com/openwrt/openwrt/pull/3087

Fixes: 33f7b80aa325 ("libopkg: drop custom md5 implementation,
unconditionally enable sha256 support")

Signed-off-by: Alexander Ryzhov <github@ryzhov-al.ru>
[Add commit description]
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
3 years agoman: remove obsolete manual pages
Baptiste Jonglez [Mon, 24 Aug 2020 14:46:00 +0000 (16:46 +0200)]
man: remove obsolete manual pages

The man pages have not been updated since 2010.  Options are documented in
the usage message obtained when running "opkg" without arguments.

In addition, the man page are no longer used anywhere in the build system
since 2017: the autoconf-based build system was removed in 6215c27b1d3a
("build: remove automake/autoconf build system").

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
3 years agoopkg_verify_integrity: better logging and error conditions
Baptiste Jonglez [Mon, 24 Aug 2020 23:00:32 +0000 (01:00 +0200)]
opkg_verify_integrity: better logging and error conditions

The function now always returns an error if size/checksum don't match: we
let the caller decide what to do with the result.

In addition, most of the logging is also moved to the caller.  We just
keep logging for unexpected errors and a bit of debug at loglevel INFO.

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
3 years agodownload: purge cached packages that have incorrect checksum
Baptiste Jonglez [Mon, 24 Aug 2020 23:00:31 +0000 (01:00 +0200)]
download: purge cached packages that have incorrect checksum

Before using a package from the cache, verify its size and checksum
against a package index, and delete the package from the cache if they
don't match.  The install process will then proceed to download the
"fixed" package as usual.

This allows to cope with remote packages that are rebuilt while keeping
the same version number as packages in the local cache.  With this change,
any outdated package in the local cache will be purged and the new version
will be downloaded instead.

This is mostly useful when running opkg on the host (e.g. in the
imagebuilder).  When running on a device, no cache is configured by
default, so this change does nothing in that case.

Fixes: FS#2690
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
3 years agodownload: factor out the logic for building cache filenames
Baptiste Jonglez [Mon, 24 Aug 2020 23:00:30 +0000 (01:00 +0200)]
download: factor out the logic for building cache filenames

If we want to access files in the cache from multiple functions, it is
necessary to have a single source of truth regarding the naming of files
in the cache.

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
3 years agolibopkg: factor out checksum and size verification
Baptiste Jonglez [Mon, 24 Aug 2020 23:00:29 +0000 (01:00 +0200)]
libopkg: factor out checksum and size verification

This is a sizeable chunk of code that be can pretty well isolated in its
own function.

This refactoring will be necessary for an upcoming feature in which
opkg_download_pkg() will need to verify the checksum of packages in the
cache.  This is the reason why the new function is located in
`opkg_download.c`.

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
3 years agodownload: remove compatibility with old cache naming scheme
Baptiste Jonglez [Mon, 24 Aug 2020 23:00:28 +0000 (01:00 +0200)]
download: remove compatibility with old cache naming scheme

A long time ago, the name of cached files was derived from the source URL.
This was changed in 2011 with d46db43e21 ("Don't include the source URI in
the cached filename.")

Some compatibility code was left behind: even today, we are still trying
to read from the old filename.  The goal of this compatibility code was to
account for existing caches that still had files with the old naming
scheme.

More than 9 years later, it is safe to remove this compatibility code.
It simplifies the download code and avoids a useless disk access.

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
3 years agolibopkg: clean up handling of unresolved dependencies
Daniel Golle [Tue, 3 Nov 2020 04:09:21 +0000 (04:09 +0000)]
libopkg: clean up handling of unresolved dependencies

Drastically improves performance, back to the level seen before
the previous commit.

Fixes: 3837489 ("libopkg: work-around yet another dependency checking problem")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Signed-off-by: Paul Spooren <mail@aparcar.org>
3 years agolibopkg: work-around yet another dependency checking problem
Daniel Golle [Mon, 2 Nov 2020 22:44:43 +0000 (22:44 +0000)]
libopkg: work-around yet another dependency checking problem

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
3 years agolibopkg: fix removal of packages
Daniel Golle [Mon, 2 Nov 2020 20:35:50 +0000 (20:35 +0000)]
libopkg: fix removal of packages

The 'dependencies_checked' field in struct abstract_pkg is also used
when removing packages, and in that case it is set to '2'.
Make the field wide enough to be able to contain that value.

Fixes: 532f92d ("libopkg: tighten length of fields in struct abstract_pkg")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
3 years agopkg_hash: remove accidentally committed stuff
Daniel Golle [Mon, 2 Nov 2020 20:30:01 +0000 (20:30 +0000)]
pkg_hash: remove accidentally committed stuff

Forgot to 'git add'...

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
3 years agolibopkg: tighten length of fields in struct abstract_pkg
Daniel Golle [Mon, 2 Nov 2020 16:43:02 +0000 (16:43 +0000)]
libopkg: tighten length of fields in struct abstract_pkg

Should reduce the size of that struct and hence memory consumption
during dependency resolution (48 bytes vs. 56 bytes on 64-bit).

Suggested-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
3 years agopkg_hash: don't swallow dependencies
Daniel Golle [Mon, 2 Nov 2020 14:35:12 +0000 (14:35 +0000)]
pkg_hash: don't swallow dependencies

Checking for unsatisfiable dependencies had the unwanted side-effect of
making opkg skip **all** dependencies. Fix that by clearing the
dependencies_checked flag after calling to
pkg_hash_fetch_unsatisfied_dependencies(), similar to how it is also
done in opkg.c.

Fixes: 8769c75 ("pkg_hash: don't suggest incompatible packages")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
3 years agopkg_hash: don't suggest incompatible packages
Paul Spooren [Tue, 27 Oct 2020 02:39:57 +0000 (16:39 -1000)]
pkg_hash: don't suggest incompatible packages

Up until now opkg would suggest packages with unsatisfied dependencies
as installable candidates. This is a frequent issue for the kmod feed in
snapshot images. In these cases opkg suggest a newer kmod version than
compatible with the installed kernel, because the same package is
available both in the kmods archive and the target specific packages
feed.

This commit fixes the issue by dropping all package problematic
candidates by checking if all their dependencies could be installed.

Signed-off-by: Paul Spooren <mail@aparcar.org>
3 years agolibopkg: fix compiler warning
Daniel Golle [Mon, 31 Aug 2020 09:55:09 +0000 (10:55 +0100)]
libopkg: fix compiler warning

libopkg/pkg.c: In function 'pkg_set_raw':
libopkg/pkg.c:109:58: warning: format '%d' expects argument of type
 'int', but argument 5 has type 'size_t' {aka 'long unsigned int'}
 [-Wformat=]

Use '%zu' instead.

Signed-off-by: Daniel Golle <daniel@makrotopia.org>
3 years agoopkg: allow to configure the path to the signature verification script
Baptiste Jonglez [Mon, 24 Aug 2020 15:07:40 +0000 (17:07 +0200)]
opkg: allow to configure the path to the signature verification script

Currently, package index signatures are only checked when opkg runs on the
OpenWrt device.  The verification script is hard-coded to a path in
/usr/sbin/.

Making this path configurable is a first step to implement signature
verification in host builds of opkg (e.g. in the imagebuilder).

Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
Acked-by: Paul Spooren <mail@aparcar.org>
3 years agolibopkg: implement lightweight package listing logic
Jo-Philipp Wich [Thu, 7 May 2020 20:19:39 +0000 (22:19 +0200)]
libopkg: implement lightweight package listing logic

Instead of building a complete package dependency tree internally, use a
lightweight list data structure to gather and sort package names, version and
descriptions.

This reduced the "opkg list" memory footprint on x86/64 from ~8MB to ~1.5MB.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
3 years agolibopkg: support passing callbacks to feed parsing functions
Jo-Philipp Wich [Sun, 3 May 2020 19:39:56 +0000 (21:39 +0200)]
libopkg: support passing callbacks to feed parsing functions

Extend pkg_hash_add_from_file(), pkg_hash_load_feeds() and
pkg_hash_load_status_files() to accept a per-package callback
function andan associated userdata pointer.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
3 years agoopkg-cl: don't read feeds on opkg update
Jo-Philipp Wich [Sun, 3 May 2020 16:24:34 +0000 (18:24 +0200)]
opkg-cl: don't read feeds on opkg update

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
3 years agolibopkg: use xsystem() to spawn opkg-key
Jo-Philipp Wich [Sun, 3 May 2020 16:01:57 +0000 (18:01 +0200)]
libopkg: use xsystem() to spawn opkg-key

Instead of the custom fork()/exec() implementation, use the existing
xsystem() helper function which provides a number of benefits:

 - It readily provides error reporting in case the execution fails
 - It has a simpler api
 - It uses vfork() internally which avoids the need to copy pages

This likely fixes https://bugs.openwrt.org/index.php?do=details&task_id=2734.

Fixes: 71f02a3 ("libopkg: add support for signature checking through usign")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
4 years agofile_util.c: refactor and fix checksum_hex2bin()
Jo-Philipp Wich [Wed, 1 Apr 2020 11:41:37 +0000 (13:41 +0200)]
file_util.c: refactor and fix checksum_hex2bin()

 - Simplify function body by reducing the amount of intermediate vars
 - Derive length checks from the size of the buf
 - Replace complex for(;;) by a simple while() advancing the src string
 - Handle edge case of a zero length input not returning NULL

Fixes: c09fe20 ("libopkg: fix skipping of leading whitespace when parsing checksums")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
4 years agofile_util.c: fix possible bad memory access in file_read_line_alloc()
Jo-Philipp Wich [Wed, 1 Apr 2020 10:41:37 +0000 (12:41 +0200)]
file_util.c: fix possible bad memory access in file_read_line_alloc()

In the case of a zero length string being returned by fgets(), the condition
checking for a trailing new line would perform a bad memory access outside
of `buf`. This might happen when line with a leading null byte is read.

Avoid this case by checking that the string has a length of at least one
byte. Also change the unsigned int types to size_t to store length values
while we're at it.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
4 years agolibopkg: fix skipping of leading whitespace when parsing checksums
Jo-Philipp Wich [Sat, 25 Jan 2020 12:57:01 +0000 (13:57 +0100)]
libopkg: fix skipping of leading whitespace when parsing checksums

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
4 years agoopkg: Fix -Wformat-overflow warning
Rosen Penev [Fri, 29 Nov 2019 21:08:32 +0000 (13:08 -0800)]
opkg: Fix -Wformat-overflow warning

A null pointer is being passed to printf. This can lead to problems.

Fixes warning:

error: '%s' directive argument is null [-Werror=format-overflow=]
   41 |    opkg_message(l, "%s: "fmt, __FUNCTION__, ##args); \
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: in expansion of macro 'opkg_msg'
   45 |  opkg_msg(l, fmt": %s.\n", ##args, strerror(errno))
      |  ^~~~~~~~
note: in expansion of macro 'opkg_perror'
  247 |   opkg_perror(ERROR, "Failed to create temporary directory '%s'",

Signed-off-by: Rosen Penev <rosenp@gmail.com>
4 years agoalternatives: remove duplicate 'const' specifier
Yousong Zhou [Fri, 14 Jun 2019 01:28:54 +0000 (01:28 +0000)]
alternatives: remove duplicate 'const' specifier

To quash warnings from GCC 7.

/home/yunion/git-repo/openwrt/openwrt/build_dir/target-mips_24kc_musl/linux-malta_be/opkg-2019-06-14-21b7bd73/libopkg/pkg_alternatives.c:35:42: warning: duplicate 'const' declaration specifier [-Wduplicate-decl-specifier]
 static const struct alternative_provider const providers[] = {
  ^~~~~

Signed-off-by: Yousong Zhou <zhouyousong@yunionyun.com>
4 years agoalternatives: special-case busybox as alternatives provider
Yousong Zhou [Wed, 12 Jun 2019 04:19:02 +0000 (04:19 +0000)]
alternatives: special-case busybox as alternatives provider

Almost all busybox applets are alternatives to some other existing
"full" utilities.  To lift the maintenance burden of enumerating CONFIG
symbols, symlink path of each applet, we special case busybox here as a
known alternatives provider.

All file pathes provided by busybox will serve as fallback alternatives
with -inf priority.  Packages intending to switch to using alternatives
mechanism will also not need to depend on the same kind of change be
applied on busybox in base system

v3 <- v2 continue to next when fetching provider pkg failed
v2 <- v1 Move busybox-specific part into data section

Signed-off-by: Yousong Zhou <zhouyousong@yunionyun.com>
Acked-by: Jo-Philipp Wich <jo@mein.io>
5 years agolibopkg: only perform size check when information is available
Jo-Philipp Wich [Thu, 31 Jan 2019 09:13:10 +0000 (10:13 +0100)]
libopkg: only perform size check when information is available

Fixes: cb66403 ("libopkg: check for file size mismatches")
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agolibopkg: check for file size mismatches
Jo-Philipp Wich [Thu, 31 Jan 2019 07:29:22 +0000 (08:29 +0100)]
libopkg: check for file size mismatches

Reject package files whose file size deviates from the size specified
in the package index in order to complicate producing hash collisions.

Ref: https://groups.google.com/d/msg/opkg-devel/o4kiGQMvkiw/hu0TVv59DgAJ
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agoRevert "libopkg: accept alternative notation for "Replaces" control field"
Jo-Philipp Wich [Fri, 18 Jan 2019 17:41:16 +0000 (18:41 +0100)]
Revert "libopkg: accept alternative notation for "Replaces" control field"

This reverts commit 1504d35f123980bf322df99fb434f6fda6e3ecd2.

We won't use Replaces: for the repo ABI_VERSION tracking after all, so
drop this hack before it becomes another liability.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agoRevert "libopkg: check installed reverse dependencies upon install/upgrade"
Jo-Philipp Wich [Fri, 18 Jan 2019 15:07:53 +0000 (16:07 +0100)]
Revert "libopkg: check installed reverse dependencies upon install/upgrade"

This reverts commit 89fe77cb119a24f728088320d66d51250a1f02ab.

After spending further thoughts on this I think this makes no sense at all
and such situations should be avoided on the repo level instead by don't
providing incompatible packages having an identical name.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agolibopkg: accept alternative notation for "Replaces" control field
Jo-Philipp Wich [Fri, 18 Jan 2019 08:01:10 +0000 (09:01 +0100)]
libopkg: accept alternative notation for "Replaces" control field

The presence of a "Replaces:" tag in any package feed list will cause
opkg prior to "d217daf libopkg: fix replacelist parsing and writing"
to segfault while parsing such a list.

Add a hack which lets opkg recognize a "Replace2" field and handle
it identically to "Replaces".

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agolibopkg: fix replacelist parsing and writing
Jo-Philipp Wich [Thu, 17 Jan 2019 21:02:37 +0000 (22:02 +0100)]
libopkg: fix replacelist parsing and writing

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agolibopkg: fix segmentation fault when traversing conflicts
Jo-Philipp Wich [Thu, 17 Jan 2019 20:37:09 +0000 (21:37 +0100)]
libopkg: fix segmentation fault when traversing conflicts

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agolibopkg: check installed reverse dependencies upon install/upgrade
Jo-Philipp Wich [Tue, 15 Jan 2019 08:19:32 +0000 (09:19 +0100)]
libopkg: check installed reverse dependencies upon install/upgrade

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agolibopkg: consider provided packages in pkg_vec_mark_if_matches()
Jo-Philipp Wich [Tue, 15 Jan 2019 07:46:15 +0000 (08:46 +0100)]
libopkg: consider provided packages in pkg_vec_mark_if_matches()

Also consider the list of provided abstract packages when testing the
package vector for name matches. This fixes the "opkg whatdepends"
command with abstract provided package arguments.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agoopkg_download: print error when fork() fails
Jo-Philipp Wich [Fri, 14 Dec 2018 07:13:38 +0000 (08:13 +0100)]
opkg_download: print error when fork() fails

Instead of silently failing and claiming that the signature verification
failed, print out the error status when the initial fork() call fails.

Ref: https://bugs.openwrt.org/index.php?do=details&task_id=1999
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
5 years agolibopkg: don't print unresolved dependencies twice
Jo-Philipp Wich [Tue, 30 Oct 2018 09:25:11 +0000 (10:25 +0100)]
libopkg: don't print unresolved dependencies twice

Sometimes opkg ends up reporting unresolved dependencies multiple
times while also missing a newline between consecutive error lines,
making the error message output looking garbled and confusing.

Add some logic to skip repeated unresolved dependencies and ensure
that message lines are properly terminated by newlines.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
6 years agoopkg_download: decode file:/ URLs
Jo-Philipp Wich [Thu, 7 Dec 2017 13:23:45 +0000 (14:23 +0100)]
opkg_download: decode file:/ URLs

Opkg might receive percent-encoded file:/ URLs from various sources,
e.g. local package lists, user-supplied command line arguments etc.

In order to support such URLs, decode the path component before
trying to invoke file operations on it.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
6 years agofile_util: implement urldecode_path()
Jo-Philipp Wich [Thu, 7 Dec 2017 13:23:00 +0000 (14:23 +0100)]
file_util: implement urldecode_path()

Introduce a new urldecode_path() helper to resolve percent-encoded URL
portions back into the original binary form.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
6 years agofile_util: consolidate hex/unhex routines
Jo-Philipp Wich [Thu, 7 Dec 2017 16:44:46 +0000 (17:44 +0100)]
file_util: consolidate hex/unhex routines

Consolidate the various copies of the bin2hex table into one global variant
and replace the broken hex() define with a static helper function to prevent
accidential multiple expression evaluation.

This also fixes bug when converting hexadecimal checksums into binary forms
where the letters a..f/A..F where incorrectly converted into 0..5 instead
of 10..15 .

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
6 years agoadd opkg option http_timeout
Alexander Couzens [Mon, 4 Dec 2017 14:41:20 +0000 (15:41 +0100)]
add opkg option http_timeout

http_timeout will append `--timeout xy` to the resulting wget
call. This is useful when using opkg update in a automatic
way and prevent long timeouts

Signed-off-by: Alexander Couzens <lynxis@fe80.eu>
6 years agoopkg: encode archive filenames while constructing download URLs
Jo-Philipp Wich [Thu, 7 Dec 2017 10:43:02 +0000 (11:43 +0100)]
opkg: encode archive filenames while constructing download URLs

Various web servers, namely Amazon S3 ones, have problems handling requests
to URLs with a literal "+" in the path component.

According to the RFC 3986 "+" is a reserved char and its purpose is
delimiting. When used in a file name it should be encoded.

Use the new urlencode_path() helper to encode the path component before
constructing the final download URL.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Tested-by: Rafał Miłecki <rafal@milecki.pl>
6 years agofile_util: implement urlencode_path() helper
Jo-Philipp Wich [Thu, 7 Dec 2017 10:40:21 +0000 (11:40 +0100)]
file_util: implement urlencode_path() helper

Introduce a helper function urlencode_path() which can be used to encode
problematic characters in package file names.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Tested-by: Rafał Miłecki <rafal@milecki.pl>
6 years agolibopkg: fix SHA256 calculation for big endian system
Jonas Gorski [Sat, 18 Nov 2017 11:30:11 +0000 (12:30 +0100)]
libopkg: fix SHA256 calculation for big endian system

The SHA256 implementation relied on WORDS_BIGENDIAN being defined for
big endian systems, but this was broken since the switch to cmake,
causing wrong checksums on big endian systems like ar71xx.

Fix this by adding a cmake check for endianess similar to the one done
by the original autotools.

Fixes: e6e6c5a21634 ("build: add cmake build system")
Acked-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Jonas Gorski <jonas.gorski@gmail.com>
6 years agopkg_parse: fix segfault when parsing descriptions with leading newlines
Jo-Philipp Wich [Fri, 28 Jul 2017 11:02:08 +0000 (13:02 +0200)]
pkg_parse: fix segfault when parsing descriptions with leading newlines

During parsing of multi-line descriptions that begin with a newline
character, the description pointer will be NULL when the first
continuation line is encountered, causing the strlen() invocation
on the description pointer to segfault.

This is caused by the fact that the parse_simple() helper used for
parsing the initial description line returns NULL instead of an
empty string when encountering only whitespace after the field name.

Rework the continuation line parsing code to not assume an initialized
pointer by checking before calling strlen() and avoiding realloc()
on a NULL pointer.

Fixes FS#933.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
6 years agopkg_alternatives: pass if the desired symlink already exists
Yousong Zhou [Tue, 11 Jul 2017 03:07:23 +0000 (11:07 +0800)]
pkg_alternatives: pass if the desired symlink already exists

This can happen when opkg installs, then configures multiple
alternatives in a batch.  Symlink to the highest prio alternative will
be created in the initial configuring stage causing later symlink call
fail with EEXIST

    Configuring busybox.
    ...
    symlink("/sbin/ip-full", "/home/yousong/j/t/lede-imagebuilder-mvebu.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl_eabi/root-mvebu/sbin/ip") = 0
    ...
    Configuring ip-full.
    ...
    symlink("/sbin/ip-full", "/home/yousong/j/t/lede-imagebuilder-mvebu.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl_eabi/root-mvebu/sbin/ip") = -1 EEXIST (File exists)
    ...

While at it, "mkdir -p" dirname(path_in_dest) before symlink in case the
following symlink call may fail with ENOENT

Ref: https://github.com/openwrt/packages/issues/4567
Reported-by: Aner Andros <aa@anerandros.info>
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
6 years agoopkg: add --no-check-certificate argument
Alexandru Ardelean [Thu, 11 May 2017 19:42:02 +0000 (22:42 +0300)]
opkg: add --no-check-certificate argument

For cases when artifacts are stored on https:// accessible
locations and you don't want to install ca-certificates
(for various reasons).

I'll admit, using SSL like this is not recommended,
but since wget (even uclient-fetch) allows the
--no-check-certificate option, it would be nice
for opkg to support setting it if needed/configured.

Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
6 years agopkg_alternatives: use ERROR level for symlink failure
Yousong Zhou [Wed, 3 May 2017 09:08:10 +0000 (17:08 +0800)]
pkg_alternatives: use ERROR level for symlink failure

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
6 years agopkg: alternatives support
Yousong Zhou [Sat, 18 Mar 2017 08:45:53 +0000 (16:45 +0800)]
pkg: alternatives support

It's a list of specs of the following form seprated by commas to describe
alternatives provided by the package

    <prio>:<path>:<altpath>

where <path> will be a symbolic link to <altpath> of the highest <prio>

Size comparison on x86_64 after the change

    function                                             old     new   delta
    pkg_alternatives_update                                -     587    +587
    pkg_parse_line                                      2101    2609    +522
    .rodata                                            24594   24738    +144
    pkg_formatted_field                                 2385    2528    +143
    pkg_deinit                                           427     486     +59
    pkg_print_status                                     264     280     +16
    opkg_configure                                        59      69     +10
    xreadlink                                            120     128      +8
    opkg_remove_pkg                                     1079    1087      +8
    ------------------------------------------------------------------------------
    (add/remove: 2/0 grow/shrink: 8/0 up/down: 1483/0)           Total: 1497 bytes

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
6 years agolibbb: xreadlink: fix memory leak on failure case
Yousong Zhou [Mon, 20 Mar 2017 02:46:15 +0000 (10:46 +0800)]
libbb: xreadlink: fix memory leak on failure case

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
7 years agopkg_run_script: use pkg->dest in half installed case
Alejandro del Castillo [Mon, 27 Mar 2017 21:58:17 +0000 (16:58 -0500)]
pkg_run_script: use pkg->dest in half installed case

Fixes running scripts for packages in half-installed state which would
otherwise fail with:

 * pkg_run_script: Internal error: ... has a NULL tmp_unpack_dir.
 * opkg_remove_pkg: not removing package "...", prerm script failed

Upstream bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=11245

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
[Jo-Philipp Wich: rebased onto opkg-lede, updated commit message]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agocheck_data_file_clashes_change: remove duplicated offline root in paths
Alejandro del Castillo [Wed, 23 Mar 2016 23:14:50 +0000 (18:14 -0500)]
check_data_file_clashes_change: remove duplicated offline root in paths

The function check_data_file_clashes_change update the ownership of
files that belong to a package being replaced. It calls
pkg_get_installed_files then adds an offline root if
opkg_conf->offline_root to each path. However, pkg_get_installed_files
already returns paths with the offline root (if present).

This fixes issue 156.

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agobuildReplaces: do not add duplicated replacees
Alejandro del Castillo [Mon, 21 Mar 2016 22:33:59 +0000 (17:33 -0500)]
buildReplaces: do not add duplicated replacees

If package A both replaces and conflicts package B, it is added
unconditionally to the replaced_by vector of package B. Add check to
only add it if it's not there to avoid inaccurate warnings of the type:

Multiple replacers for X, using first one

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agoopkg: compare-versions doesn't need any state
Ross Burton [Tue, 1 Mar 2016 16:54:26 +0000 (16:54 +0000)]
opkg: compare-versions doesn't need any state

compare-versions doesn't need lock files created and databases opened, so don't.

Signed-off-by: Ross Burton <ross.burton@intel.com>
7 years agoopkg_cmd: fix segmentation fault in opkg_compare_versions_cmd()
Jo-Philipp Wich [Wed, 15 Mar 2017 02:08:18 +0000 (03:08 +0100)]
opkg_cmd: fix segmentation fault in opkg_compare_versions_cmd()

Due to the fact that we're using blob buffers internally now we cannot simpyl
call parse_version() on an uninitialized struct.

Properly create the temporary packages using pkg_new() and deallocated them
using pkg_deinit() and free().

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agoopkg_cmd: fix return of opkg compare-versions
Ross Burton [Tue, 1 Mar 2016 18:03:29 +0000 (18:03 +0000)]
opkg_cmd: fix return of opkg compare-versions

The implementation of opkg compare-version was simply returning the return value
of pkg_version_satisfied(), which returns a boolean.  This results in inverted
logic in that:

$ opkg compare-version 1.0 == 1.0 || echo different
different

That's clearly not right.  As nobody had noticed this before presumably nobody
was actually using this (likely either opkg-compare-version or
opkg-compare-versions.sh) just change the return value so it's correct.

Signed-off-by: Ross Burton <ross.burton@intel.com>
7 years agopkg_src_list_push: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:37 +0000 (16:27 -0600)]
pkg_src_list_push: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agopkg_src_list_pop: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:36 +0000 (16:27 -0600)]
pkg_src_list_pop: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agostr_list_push: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:35 +0000 (16:27 -0600)]
str_list_push: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agostr_list_prev: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:34 +0000 (16:27 -0600)]
str_list_prev: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agostr_list_last: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:33 +0000 (16:27 -0600)]
str_list_last: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agoconffile_list_pop: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:28 +0000 (16:27 -0600)]
conffile_list_pop: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agopkg_dest_list_push: remove unused functions
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:31 +0000 (16:27 -0600)]
pkg_dest_list_push: remove unused functions

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agonv_pair_list_prev: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:30 +0000 (16:27 -0600)]
nv_pair_list_prev: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agonv_pair_list_last: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:29 +0000 (16:27 -0600)]
nv_pair_list_last: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agoconffile_list_pop: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:28 +0000 (16:27 -0600)]
conffile_list_pop: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agoconffile_list_push: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:27 +0000 (16:27 -0600)]
conffile_list_push: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agoactive_list_sort: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:26 +0000 (16:27 -0600)]
active_list_sort: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
[Jo-Philipp Wich: remove call from opkg_active_list_test]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agoactive_list_add_depend: remove unused function
Alejandro del Castillo [Wed, 10 Feb 2016 22:27:25 +0000 (16:27 -0600)]
active_list_add_depend: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
[Jo-Philipp Wich: remove call from opkg_active_list_test]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agopkg_dependence_satisfied: remove unused function
Alejandro del Castillo [Mon, 23 Nov 2015 19:48:42 +0000 (13:48 -0600)]
pkg_dependence_satisfied: remove unused function

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agoFix typo in pkg_hash_fetch_best_installation_candidate.
Michael Hansen [Wed, 14 Sep 2016 20:42:26 +0000 (13:42 -0700)]
Fix typo in pkg_hash_fetch_best_installation_candidate.

A (presumed) typo in pkg_hash_fetch_best_installation_candidate caused the
matching_pkgs vec to be sorted twice, instead of sorting matching_apkgs.
This fixes both the call and the API to match what appears to be intended.

Signed-off-by: Michael Hansen <michael.j.hansen@hp.com>
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
7 years agoset_flags_from_control: remove function
Alejandro del Castillo [Wed, 22 Feb 2017 23:32:08 +0000 (17:32 -0600)]
set_flags_from_control: remove function

During installs, install_data_files calls set_flags_from_control to
capture the case where fields, such as "Essential", are present in the
package control file but are missing on the Packages file. This
operation is expensive (re-parses the control file) and is catering to a
case that should not be supported on the first place.

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
[Jo-Philipp Wich: remove function prototype as well]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agoopkg_active_list_test: mark functions static, swap strcmp() with memcmp()
Jo-Philipp Wich [Wed, 15 Mar 2017 00:32:52 +0000 (01:32 +0100)]
opkg_active_list_test: mark functions static, swap strcmp() with memcmp()

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibopkg_test: mark functions static, remove unused opkg_test()
Jo-Philipp Wich [Wed, 15 Mar 2017 00:29:35 +0000 (01:29 +0100)]
libopkg_test: mark functions static, remove unused opkg_test()

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agopkg_parse: fix bad message level for opkg_msg() call
Jo-Philipp Wich [Wed, 15 Mar 2017 00:26:16 +0000 (01:26 +0100)]
pkg_parse: fix bad message level for opkg_msg() call

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agopkg_parse: remove unused variable in pkg_parse_line()
Jo-Philipp Wich [Wed, 15 Mar 2017 00:23:31 +0000 (01:23 +0100)]
pkg_parse: remove unused variable in pkg_parse_line()

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agopkg_parse: remove const requirement from pkg_parse_line()
Jo-Philipp Wich [Wed, 15 Mar 2017 00:21:03 +0000 (01:21 +0100)]
pkg_parse: remove const requirement from pkg_parse_line()

To simplify parsing we'd like to be able to modify the passed line buffer
inplace, so remove the const requirement from the pkg_parse_line() argument.

Convert pkg_parse_line's sole user accordingly.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agopkg_hash: remove unused variable in pkg_hash_load_feeds()
Jo-Philipp Wich [Wed, 15 Mar 2017 00:16:23 +0000 (01:16 +0100)]
pkg_hash: remove unused variable in pkg_hash_load_feeds()

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agopkg_depends: add missing parse_replacelist() prototype
Jo-Philipp Wich [Wed, 15 Mar 2017 00:13:26 +0000 (01:13 +0100)]
pkg_depends: add missing parse_replacelist() prototype

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agoopkg_utils: add missing header
Jo-Philipp Wich [Wed, 15 Mar 2017 00:11:14 +0000 (01:11 +0100)]
opkg_utils: add missing header

Include "opkg_utils.h" header file to declare function prototypes.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibopkg: remove unused progress parameters from opkg_download()
Jo-Philipp Wich [Wed, 15 Mar 2017 00:04:49 +0000 (01:04 +0100)]
libopkg: remove unused progress parameters from opkg_download()

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibopkg: remove unused progress callback
Jo-Philipp Wich [Tue, 14 Mar 2017 23:59:45 +0000 (00:59 +0100)]
libopkg: remove unused progress callback

The current opkg_download() implementation completely ignores the passed
progress callback function, so drop this unused code.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibopkg: remove unused variable from opkg_install_package()
Jo-Philipp Wich [Wed, 15 Mar 2017 00:06:32 +0000 (01:06 +0100)]
libopkg: remove unused variable from opkg_install_package()

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibbb: remove unused variables
Jo-Philipp Wich [Tue, 14 Mar 2017 23:52:47 +0000 (00:52 +0100)]
libbb: remove unused variables

Remove unused variables from deb_extract() to avoid compiler warnings.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibbb: change gzip_read() buffer argument to void *
Jo-Philipp Wich [Tue, 14 Mar 2017 23:50:46 +0000 (00:50 +0100)]
libbb: change gzip_read() buffer argument to void *

Make gzip_read() expect a void * buffer argument to avoid signed vs unsigned
char mismatch warnings.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibbb: fix gzip_thread() warnings
Jo-Philipp Wich [Tue, 14 Mar 2017 23:47:23 +0000 (00:47 +0100)]
libbb: fix gzip_thread() warnings

 - mark function static
 - add missing return statement
 - avoid uninitialized use of "len"

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibopkg/opkg_cmd.c: more robust PATH handling
Glenn Strauss [Thu, 2 Mar 2017 22:53:42 +0000 (17:53 -0500)]
libopkg/opkg_cmd.c: more robust PATH handling

preserve semantics of PATH when PATH is not set in environment

error and undefined behavior reported in
  https://github.com/openwrt/packages/issues/1922
when PATH not set in environment
(lighttpd executes CGI with empty base env, plus standard CGI env vars)

Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>
[Jo-Philipp Wich: avoid free() on NULL, use default from cmake cache string]
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
7 years agolibopkg: specify "/bin/sh" instead of "sh"
Glenn Strauss [Thu, 2 Mar 2017 22:53:09 +0000 (17:53 -0500)]
libopkg: specify "/bin/sh" instead of "sh"

avoid strange behavior with execvp() when PATH is not set in environment
(in which case confstr(_CS_PATH) should return something reasonable)

reproducable running openwrt 15.05 and 15.05.1 and attempting to install
a software package (e.g. libuuid) via LuCI (prior to openwrt/luci#1048).
(https://github.com/openwrt/luci/pull/1048) libuuid.postinst fails with
status 255 on 15.05 and opkg segfaults in 15.05.1.  This probably merits
further exploration.

Originally reported in https://github.com/openwrt/packages/issues/1922

Signed-off-by: Glenn Strauss <gstrauss@gluelogic.com>
7 years agounarchive: fix possible segmentation fault in deb_extract()
Jo-Philipp Wich [Tue, 28 Feb 2017 23:46:39 +0000 (00:46 +0100)]
unarchive: fix possible segmentation fault in deb_extract()

When a not existing or unreachable file path is passed to deb_extract(),
the wfopen() call fails, causing a jump to the cleanup: label which leads
to a call to gzip_close() on the tar_outer structure.

The tar_outer structure however contains uninitialized memory at this point,
causing gzip_close() to operate on garbage data. Depending on the nature of
the unitialized memory, this might lead to all sorts of issues, e.g. freeing
of not allocated memory or invoking fclose() on garbage pointers.

Solve this problem by initializing the tar_outer and tar_inner structures
right at the declaration.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>