Amiga.org

Amiga News and Community Announcements => Amiga News and Community Announcements => Amiga Software News => Topic started by: patrik on December 18, 2019, 08:09:02 PM

Title: New UHC Tools release
Post by: patrik on December 18, 2019, 08:09:02 PM
UHC Tools has been around for a while now, but I don't think that many people know about it.

In short it is a collection of utilities and scripts to make your Amiga more powerful. Most parts of UHC Tools require a network or internet connection to do this.

The main feature is what we call the searchers. The most popular searchers are aminetsearch, fishsearch, modlandsearch and whdloadsearch, but there are more. Each searcher generally consist of a search script, an extract script and a readme script, but some have more.

To give an example, lets say you are looking for the latest SysInfo version on aminet. Then you can use aminetsearch like this:

8.Ram Disk:> aminetsearch sysinfo
# Filename            Path      Size Description
0 SysInfo101.lha      comm/cnet   6K SysInfo V1.01 Shows info system
1 Sysinfo_Amiplug.lha comm/irc    7K Plugin for AmIRC to show system specs.
2 SysInfo_Oberon.lha  dev/obero  16K AO Interface for SysInfo.library v1.30
3 SysInfo.lha         util/moni  53K Classic Amiga System Information Program
4 SysInfo40_noFPU.lha util/moni   1K No FPU version of SysInfo 4.0
5 batsi.lha           util/moni   4K Add your expansion boards to SysInfo


The important thing to note is the leftmost index number, which is then used with aminetreadme to read the more detailed description, or together with aminetextract to download and extract the archive:

8.Ram Disk:> aminetextract 3
Downloading de.aminet.net/aminet/util/moni/SysInfo.lha
Receiving file: 0% - [oooooooooooooooooooooooooooooooo] - 100% (52.93kB)
Extracting SysInfo.lha to Ram Disk:
8.Ram Disk:> list sysinfo#?
SysInfo                      Dir ----rwed Today     20:55:16
SysInfo.info                 740 ----rwed 14-Aug-19 21:53:10
1 file - 1 directory - 4 blocks used


So very quick and easy to search, read readmes, make a decision and then the download and extract is done in one operation.

There are also other very useful utilities, like SynergyClient (http://uhc.driar.se/help/SynergyClient) which makes it possible to control your Amiga keyboard and mouse input from another computer, like the Amiga was one of its screens.

UHC Tools is available for AmigaOS 2.04+, MorphOS 1.4+, AmigaOS4+ and AROS i386.

Details on how to install it and what exactly is included can be found at http://uhc.driar.se/ or http://aminet.net/package/util/misc/UHCTools.
Title: Re: New UHC Tools release
Post by: kolla on December 19, 2019, 09:52:51 AM
One small request, similar to the one I had before about supporting $http_proxy for aget... I presume UHCSearch also only talks http? Could UHCSearch also support $http_proxy?
Title: Re: New UHC Tools release
Post by: dschallock on December 19, 2019, 10:38:00 PM
This is awesome.  Thank you for working on this project.  I think its important for Amiga users in 2019/2020 that use their amiga's on the internet, to have new ways of accessing information and data from the web beyond just web browsers.  I am super thankful for the latest version of ibrowse and it is a fantastic piece of software and I bought the new version, but coders writing new apps that are tailored for the Amiga and allow people to access more from the net through those specific programs is a great thing.
Has anyone thought about making a GUI front end for these tools?
Title: Re: New UHC Tools release
Post by: kolla on December 20, 2019, 09:25:30 AM
Yes, even before UHCTools I had some scripts that essentially built an amiga guide interface to Aminet, but I was “cheating” by using rsh+zsh-function for doing the “aminet2guide” parsing/conversion on a linux system, saving time and resources on the Amiga. When I became aware of UHCTools I rewrote my scripts to use UHCTools and do the conversion to amigaguide on the Amiga, quick&dirty, using ThoR’s SED... it should really be done just using arexx though :)

The “obstacles” for my amigaguide interface are...
* lack of OS standard RequestString for getting user input (most importantly to do search) - there are a bunch of them, all with different syntax, all relying on reqtools.library (sigh!) and there is no way to use Multiview’s built-in search as it is not programmable (quadrupled sigh!)
* the aget program in UHCTools only has a progress bar for terminal console, no “GUI” progressbar.

It’s been a while since I worked on it, but I should really “appify” what I have and just make it available for others to hack on :)

And yes - thanks a lot to the UHCTools team - awesome project!
Title: Re: New UHC Tools release
Post by: patrik on December 21, 2019, 04:24:33 PM
@kolla:

UHCSearch does not use http. The protocol used is much simpler and consistent when compared to http, but it is a request/response protocol. It is a possible future change though as there is at least one undeniable benefit of using http that cannot be disregarded - outgoing connections to port 80 is generally never blocked in anyones firewall.

It should be possible to send the UHCSearch traffic through a http proxy by using the CONNECT method. May I ask for some motivational use-case? =).

Regarding progress bar, I have been thinking about improving the aget progress bar in general. It is actually not a bad idea to optionally be able to show a "GUI" progress bar.

Has the 3.2 team been notified about the great RequestString suggestion and the Multiview search limitations?


@dschallock:

There is no GUI available, but it is still very powerful in its current form. Observe that the searchers will require a match for all words provided, so if you get too big result list, you can abort scrolling with ctrl-c and add more search words to cut down the number of results.

One thing I also can recommend is to set the UHCREADER ENV variable to say Multiview or MuchMore. This will make the the searcher's readme commands (aminetreadme, fishreadme, etc) display the readmes in a separate window, both increasinging control over reading and not scrolling away the search results.
Title: Re: New UHC Tools release
Post by: kolla on December 21, 2019, 10:03:12 PM
May I ask for some motivational use-case? =)

Of course :)

The use case is when the Amiga system's isn't really online, but only is on a IPv4 peer-to-peer link to a modern system, that is on an IPv6-only LAN, using NAT64/DNS64 to access the legacy IPv4-only services - such as the UHC search sites - perhaps not a common scenario today, but one that may become more and more common in time ;)

Quote
Has the 3.2 team been notified about the great RequestString suggestion and the Multiview search limitations?
Yes, silence regarding RequestString, and a plain "no" on the Multiview search, it simply isn't available for Amiga guide datatype, and it isn't programmable.

Quote
One thing I also can recommend is to set the UHCREADER ENV variable to say Multiview or MuchMore.
I looked into my ENVARC: the other day, so much UHC#? :)
Perhaps an idea to put them in a folder? ${UHC/VIEWER} for example..

Damn, I now found another bug in OS 3.1.4 (and earlier)...
Code: [Select]
Setenv UHC/VIEWER Multiview
It creates ENV:UHC folder and the ENV:UHC/VIEWER file with correct content.

However
Code: [Select]
SetEnv SAVE UHC/VIEWER Multivew
does not create ENVARC:UHC folder and ENVARC:UHC/VIEWER file...

Title: Re: New UHC Tools release
Post by: patrik on December 22, 2019, 11:51:53 AM
The use case is when the Amiga system's isn't really online, but only is on a IPv4 peer-to-peer link to a modern system, that is on an IPv6-only LAN, using NAT64/DNS64 to access the legacy IPv4-only services - such as the UHC search sites - perhaps not a common scenario today, but one that may become more and more common in time ;)

Sounds like a very dark future, I do hope we as a community have an IPV6 capable TCP/IP stack at that time. However, I do see the point, good to be prepared.

Quote
I looked into my ENVARC: the other day, so much UHC#? :)
Perhaps an idea to put them in a folder? ${UHC/VIEWER} for example..

We are aware, the plan is to tidy this up.

Quote
Damn, I now found another bug in OS 3.1.4 (and earlier)...
Code: [Select]
Setenv UHC/VIEWER Multiview
It creates ENV:UHC folder and the ENV:UHC/VIEWER file with correct content.

However
Code: [Select]
SetEnv SAVE UHC/VIEWER Multivew
does not create ENVARC:UHC folder and ENVARC:UHC/VIEWER file...

I have reported this bug, thanks for finding it.
Title: Re: New UHC Tools release
Post by: nbache on December 22, 2019, 12:14:59 PM
@patrik:

Note that this same bug was fixed some time ago in OS4 - in case that is useful for the 3.2 devs.

Best regards,

Niels
Title: Re: New UHC Tools release
Post by: kolla on December 22, 2019, 05:14:15 PM
The use case is when the Amiga system's isn't really online, but only is on a IPv4 peer-to-peer link to a modern system, that is on an IPv6-only LAN, using NAT64/DNS64 to access the legacy IPv4-only services - such as the UHC search sites - perhaps not a common scenario today, but one that may become more and more common in time ;)

Sounds like a very dark future, I do hope we as a community have an IPV6 capable TCP/IP stack at that time.

Well, some of us have been living this "dark future" for quite some time (years) already. It's not so dark really, the only issue is legacy IPv4-only systems, but they are few and far apart. Aside from Amiga, the only other I can think of is Nintendo systems, but their "Wii" network is long gone anyways, right? :)

We as a community have been stuck with our heads in the sand for 2 decades, even as other alternative operating systems like RiscOS, Haiku, ReactOS etc have at least recognised the problem, have roadmaps that include IPv6, and even IPv6 implemented - for Amiga systems, there is nada... closest was Holger Kruse when he worked on Miami, he _had_ IPv6 plans. Two decades ago. So, I don't have much hopes for an Amiga IPv6 stack, really, everyone is just hoping for some magic router that will fix it all.
Title: Re: New UHC Tools release
Post by: patrik on December 26, 2019, 02:39:12 PM
@kolla:

So if you want to use UHCSearch as it is now, how do you do it?
Title: Re: New UHC Tools release
Post by: kolla on December 27, 2019, 12:03:52 AM
@kolla:
So if you want to use UHCSearch as it is now, how do you do it?

At home, I general, I don't.

I have my own aminet mirror and an "aminet search" is rsh+ssh to the BSD box that holds the mirror, via a raspberry pi, where it just performs a grep on the INDEX file :)

But I have my Minimig and MiSTer at work, where I've used UHCSearch as intended.

A work-around for my home-situation is to make an ipv4-tunnel from the home Amiga systems to some cloud service, for example (though these days some providers offer cheaper VPS instances when you make them IPv6-only...)
Title: Re: New UHC Tools release
Post by: patrik on December 30, 2019, 03:38:49 PM
I looked into my ENVARC: the other day, so much UHC#? :)
Perhaps an idea to put them in a folder? ${UHC/VIEWER} for example..

Code: [Select]
8.Ram Disk:> uhcstatus
UHC Revision: 4938182ec00a058d1b126b02a8b3a53218cdf600

Type:             ENV:               Value:
Location          UHCBIN             SYS:UHC/
Temp Location     UHC/TEMPDIR        T:
Reader            UHC/READER         RES MuchMore
Show Archiver     UHC/SHOWARCHIVER   No
Show DL Progress  UHC/SHOWPROGRESS   Yes
Aminet Mirror     UHC/AMINETMIRROR   de.aminet.net/aminet/

A bit tidier?
Title: Re: New UHC Tools release
Post by: kolla on December 30, 2019, 04:23:37 PM
A bit tidier?
Yes, very cool, thanks :)
Title: Re: New UHC Tools release
Post by: patrik on January 06, 2020, 03:28:15 PM
@kolla:
So if you want to use UHCSearch as it is now, how do you do it?

At home, I general, I don't.

I have my own aminet mirror and an "aminet search" is rsh+ssh to the BSD box that holds the mirror, via a raspberry pi, where it just performs a grep on the INDEX file :)

But I have my Minimig and MiSTer at work, where I've used UHCSearch as intended.

A work-around for my home-situation is to make an ipv4-tunnel from the home Amiga systems to some cloud service, for example (though these days some providers offer cheaper VPS instances when you make them IPv6-only...)

We have changed UHCSearch to use HTTP, so now it should be as firewall friendly as possible.

It also supports sending the requests via a HTTP proxy by using the http_proxy ENV-variable.

Please report any issues you find.
Title: Re: New UHC Tools release
Post by: kolla on January 07, 2020, 06:09:15 AM
We have changed UHCSearch to use HTTP, so now it should be as firewall friendly as possible.

It also supports sending the requests via a HTTP proxy by using the http_proxy ENV-variable.

Please report any issues you find.

Very cool, it works well for me now :)
Title: Re: New UHC Tools release
Post by: kolla on January 07, 2020, 06:24:40 AM
Code: [Select]
8.Ram Disk:> uhcstatus
UHC Revision: 4938182ec00a058d1b126b02a8b3a53218cdf600

Type:             ENV:               Value:
Location          UHCBIN             SYS:UHC/
Temp Location     UHC/TEMPDIR        T:
Reader            UHC/READER         RES MuchMore
Show Archiver     UHC/SHOWARCHIVER   No
Show DL Progress  UHC/SHOWPROGRESS   Yes
Aminet Mirror     UHC/AMINETMIRROR   de.aminet.net/aminet/

I noticed that aminetget and aminetextract now always resets ${UHC/AMINETMIRROR} to "de.aminet.net/aminet/" - the culprit being
Code: [Select]
IF "$UHC/AMINETMIRROR" EQ "*$UHC/AMINETMIRROR", which of course should be
Code: [Select]
IF "${UHC/AMINETMIRROR}" EQ "*${UHC/AMINETMIRROR}" like it is in aminetreadme

When debugging the above issue, I noticed that the SetFromCmd fails (or rather, creates unwanted output so that uhcget fails), when the shell is running with "set ech on" - perhaps a long talk with Thomas Richter about stdout vs stderr can enlighten you on how this can be fixed :)
Title: Re: New UHC Tools release
Post by: patrik on January 07, 2020, 07:16:36 AM

I noticed that aminetget and aminetextract now always resets ${UHC/AMINETMIRROR} to "de.aminet.net/aminet/" - the culprit being
Code: [Select]
IF "$UHC/AMINETMIRROR" EQ "*$UHC/AMINETMIRROR", which of course should be
Code: [Select]
IF "${UHC/AMINETMIRROR}" EQ "*${UHC/AMINETMIRROR}" like it is in aminetreadme

Fixed, thank you very much.

When debugging the above issue, I noticed that the SetFromCmd fails (or rather, creates unwanted output so that uhcget fails), when the shell is running with "set ech on" - perhaps a long talk with Thomas Richter about stdout vs stderr can enlighten you on how this can be fixed :)

Interesting - before the "set echo on" feature, I have not seen a use for not capturing everything. Right now SetFromCmd swallows everything to be consistent on all platforms, so it for example can be used to not show unwanted output on failure also on OS4, which we cannot mute otherwise (*>NIL: stderr redirection cannot be used, as that would break the scripts on for example 3.1).
Title: Re: New UHC Tools release
Post by: kolla on January 08, 2023, 08:13:08 AM
A bug report…

HTTP proxy by using the http_proxy ENV-variable.

This stopped working with "AsyncHTTPGet 0.84 ( 7.jan.23)", I now get "10.99.0.1:8888: Host name lookup failure (2)" when trying to download anything. Previous aget works fine.

Also a suggestion, in UHC-Startup…

Replace…

List UHC:UHC-User-Startup/ LFORMAT="Execute ${UHCBIN}UHC-User-Startup/%N" >T:UHC-User-Startup.temp

with (for example)…

List UHC:UHC-User-Startup FILES PAT "~(#?.info)" LFORMAT="Execute *"%P%N*"" >T:UHC-User-Startup.temp


Some editors leave .info files, and Execute doesn’t handle it well when told to execute them.
(and I don’t quite grasp why there would be any reason for not using %P?)
Title: Re: New UHC Tools release
Post by: patrik on January 16, 2023, 06:04:38 PM
A bug report…

This stopped working with "AsyncHTTPGet 0.84 ( 7.jan.23)", I now get "10.99.0.1:8888: Host name lookup failure (2)" when trying to download anything. Previous aget works fine.

Thank you for reporting that so quickly! Fixed 2023-01-11, but could not respond until today.

Also a suggestion, in UHC-Startup…

Replace…

List UHC:UHC-User-Startup/ LFORMAT="Execute ${UHCBIN}UHC-User-Startup/%N" >T:UHC-User-Startup.temp

with (for example)…

List UHC:UHC-User-Startup FILES PAT "~(#?.info)" LFORMAT="Execute *"%P%N*"" >T:UHC-User-Startup.temp


Some editors leave .info files, and Execute doesn’t handle it well when told to execute them.
(and I don’t quite grasp why there would be any reason for not using %P?)

Great suggestion, I have changed that now :).
Title: Re: New UHC Tools release
Post by: kolla on March 25, 2023, 07:40:34 AM
Hello Patrik!

For a few weeks now, updates in Aminet has not been showing, that is, aminetrecent seem "stuck" with fe.lha as latest upload, and that was 2023-03-02. I've just been taking it for granted that someone else would report this, cannot believe I'm the only one who regularly check for aminet updates using UHC :)

Using latest uhc-tools (according to uhcupdate)
I see that aminetrecent uses ${UHC/SEARCHURL}, which is uhc.driar.se/uhcsearch - if I remove the var, an "aminetrecent" reinstates it.
Title: Re: New UHC Tools release
Post by: patrik on April 01, 2023, 09:12:23 AM
Hello Patrik!

For a few weeks now, updates in Aminet has not been showing, that is, aminetrecent seem "stuck" with fe.lha as latest upload, and that was 2023-03-02. I've just been taking it for granted that someone else would report this, cannot believe I'm the only one who regularly check for aminet updates using UHC :)

Using latest uhc-tools (according to uhcupdate)
I see that aminetrecent uses ${UHC/SEARCHURL}, which is uhc.driar.se/uhcsearch - if I remove the var, an "aminetrecent" reinstates it.

Fixed, thank you very much for the report!

The reinstating is intentional, it is designed to be able to sort itself out so to say. Both search mirrors were affected by the issue btw, so using the next uhc.megaburken.net/uhcsearch would not have helped.
Title: Re: New UHC Tools release
Post by: kolla on April 04, 2023, 12:25:11 AM
What if uhc-tools came with a simlle (eh) tool for quickly reporting bugs/problems? :)