misc/collect.py: introduce variable {base} to fix download_url

While scanning a directory for profile.json files of different releases,
the releases might be in different sub directories. But the collect.py
script only takes one download URL template for all overview.json files.

This commit introduces a variable {base} for the download URL parameter
that will be replaced by the sub directory that distinguish the location
of different release from each other.

Example path structure:
* ../tmp/releases/18.06.8/targets
* ../tmp/snapshot/targets

collect.py call:
```
./misc/collect.py scan "https://download.openwrt.org/{base}/{target}" ../tmp/ www/
```
In www/config.json:
```
versions: {'18.06.8': 'data/18.06.8/overview.json', 'SNAPSHOT': 'data/SNAPSHOT/overview.json'},
```

In www/data/18.06.8/overview.json:
```
"download_url": "https://download.openwrt.org/releases/18.06.8/targets/{target}",
```

In www/data/SNAPSHOT/overview.json:
```
"download_url": "https://download.openwrt.org/snapshots/targets/{target}"
```

diff --git a/misc/collect.py b/misc/collect.py
index fbaa4ceab8390083934101b27d0efcb9962a1f29..527d27a5bfe7a2e2e85e1a340bf3b17e7ddebb47 100755 (executable)
--- a/misc/collect.py
+++ b/misc/collect.py
@@ -329,8 +329,30 @@ def scan(args):
                 }
             )
 
+    """
+        Replace {base} variable in download URL with the intersection
+        of all profile.json paths. E.g.:
+        ../tmp/releases/18.06.8/targets => base is releases/18.06.8/targets
+        ../tmp/snapshots/targets => base in snapshots/targets
+    """
+
+    def replace_base(releases, target_release, download_url):
+        if "{base}" in download_url:
+            # release => base path (of profiles.json locations)
+            paths = {}
+            for release, profiles in releases.items():
+                paths[release] = os.path.commonpath(profiles.keys())
+            # base path of all releases
+            release_path_base = os.path.commonpath(paths.values())
+            # get path intersection
+            base = str(paths[target_release])[len(release_path_base) + 1 :]
+            return download_url.replace("{base}", base)
+        else:
+            return download_url
+
     for release, profiles in releases.items():
-        output = merge_profiles(profiles, args.download_url)
+        download_url = replace_base(releases, release, args.download_url)
+        output = merge_profiles(profiles, download_url)
 
         versions[release] = "data/{}/overview.json".format(release)
         os.makedirs("{}/{}".format(data_path, release), exist_ok=True)