lame benchmarks (pun intended)

[Piru]

AmigaONE X1000 has been benchmarked against Mac mini G4 1.5GHz and AmigaONE 500 460EX 1.15GHz with lame (mp3 encoder). The graph shows some interesting results:

X1000 appears to crush the competition, even with only single core enabled.

Except that interestingly the MorphOS results are from a lame build without AltiVec SIMD support. Why is that significant you may ask? The AltiVec accelerated version is significantly faster than the scalar (non-AltiVec) one. See what happens when you run the tests with the proper AltiVec enabled lame:

[Piru]

Piru, also could you link the source audio file you used in the conversion?

http://sintonen.fi/temp/AKsack.wav (md5sum: 5f644fbe4fe60211f8f1dde60e55dd53)

To benchmark it, download the file, and then issue the following command:

Code: [Select]

lame AKsack.wav
(with MorphOS make sure you're using the lame_vmx binary if you have altivec!)

[Piru]

@Piru

According to the lame readme on os4depot, the AmigaOS4 version also lacks altivec support (the porter couldn't get it working properly). The A1-X1000's CPU has an altivec unit.

Hans

The benchmark author claims he used the altivec version (I can only presume he used the older Stephan Rupprecht's version included in the said archive: lame-398-4/bin/lame.g4-3.98.2)

In english this post says: "Of course, why should I use other version while my CPU has AltiVec. Don't ask me about MorphOS, but I assume that they also used AV lame on G4 machines"

As far as I remember the PA6-T AltiVec indeed is slower than the one found from E600 (G4). It's somewhere close to 970 (G5) IIRC. Per clock the e600 altivec is the fastest implementation to date.

[Piru]

The PPC Macs are outdated hardware from 2006

The 10+ year old 500MHz PowerMac is beating the AmigaOne 500, too. Oh the humanity!

[Piru]

@piru: ..has it now been confirmed that the same file has been used for the test??

The "Mac mini G4 1.5GHz" 33 second result is from a test that used the AKsack.wav.

This file (and the Mac mini G4 33 sec result) are derived from an earlier benchmark:
http://www.apc74.ppa.pl/PPA/Efika_vs_reszta_swiata.html#table7 and
http://www.apc74.ppa.pl/PPA/Efika_vs_reszta_swiata.html#table8

In this old benchmark non-altivec lame was utilized.

[Piru]

Still dunno how 1.8ghz can be the same as 1.4ghz by tests. Something wrong somethere still.

It can happen if PA6-T AltiVec isn't as fast as the E600 (7447/7448) one. Considering PA-Semi didn't have the freescale core to derive their work on, this may well be the case.

Also, mp3 encoding is highly CPU bound task, and thus a fast bus doesn't help much. Clearly PA6-T will excel in anything that is memory bound (as we've seen the RageMem and stream benchmarks).

[Piru]

Sorry if I've overlooked it, but can you confirm if the first graph and second graph are based on results carried out under the same conditions?

I have no way of knowing how the X1000 and AmigaOne 500 results were derived, except the file and lame options (none) being used. The author claims the X1000 result is from altivec accelerated lame (but I have no way of verifying the claim).

What I did was to run the MorphOS G4 results with altivec enabled lame.

If that's the case, it suggests that for the PASemi either altivec was enabled in both tests or in neither test. The time was the same in both instances (18s). There should be some difference if one was scalar and the other vector, surely?

The X1000 result is assumed to be altivec already (as claimed by the author).

[Piru]

Its unpossible that they will be slower than on sam460 (or the same).

PA6-T is easily faster than 460ex in scalar operations, no question about that.

[Piru]

@Piru

According to the lame readme on os4depot, the AmigaOS4 version also lacks altivec support (the porter couldn't get it working properly).

You can easily verify that the lame.g4-3.98.2 binary contains altivec support:

• > wget http://os4depot.net/share/audio/record/lame.lha
  
• > lha x lame.lha
• > ppc-amigaos-objdump -d lame-398-4/bin/lame.g4-3.98.2 | egrep v[0-9]+,

[Piru]

Doesn't mean much. I also have some stuff which have altivec insturctions inside, but still, that piece of code unused in the necessary place. In the readme to that archive a lot to say about mess with altivec, so it can be not suprise that its just broken, disabled and not works.

Did you read the readme? Here's the relevant part highlighted by me:

Unfortunately the people who tested the Altivec version for me did not have
any success, and therefore this upload only includes a generic PPC version
of 3.98.4, and Stephan Rupprecht's prior 3.98.2 G4 build.

The archive has 3 lame binaries:

Code: [Select]

[generic]               197665  411156  48.1% -lh5- 3de4 Oct  6  2010 lame-398-4/bin/lame
[generic]                50969  168764  30.2% -lh5- 9a1e Oct  6  2010 lame-398-4/bin/lame-shared
[generic]               322281  710768  45.3% -lh5- 2952 Sep 26  2008 lame-398-4/bin/lame.g4-3.98.2
The first two are the non-altivec builds (one static and the other using SObjs/libmp3lame.so).

The third is the old (26-sep-2008) - altivec enabled - build by Stephan Rupprecht, included as-is.

The same is confirmed here as well:

In any case, no-one loses anything. 3.98.4 for generic PPC, new libmp3lame which was always generic, and the prior 3.98.2 for altivec is in here. Win-win.

[Piru]

@Piru

I tryed all the lame versions on hte archive from OS4Depot and get:

for Lame-G4 : 23 secs

for lame/lameSHARED : 40 secs

Indeed. Now, would Hans_ & kas1e please figure this thing out already? OS4 *does* have altivec enabled lame.

[Piru]

@Piru

the problem was only if the X1000 test author have used the G4 version...

Why else would he claim he did use the altivec version?

[Piru]

@Hans_

Thank you. Nothing wrong with the results as we can see.

Also, the scalar results are rather interesting. Here's the result from my Mac mini G4 1.5GHz (first scalar, then altivec):

Code: [Select]

2·Ram Disk:T% showconfig
SYSTEM:     PowerMac10,2
REGISTERED: Harry Sintonen
PROCESSOR:  7447A (G4) (V1.5) 1500MHz (FSB 166MHz)
VERSION:    MorphOS version 2.7, Ambient version 1.43, Kickstart version 51.37
RAM:        Node type $8A, Attributes $1505 (FAST), at $20000CA8-$5CD8D000 (~973.5 meg)
BOARDS:
 Vendor $106B Device $0034 Apple Computer Inc., UniNorth 2 AGP
 Vendor $1002 Device $5962 ATI Technologies Inc, RV280 [Radeon 9200]
 Vendor $14E4 Device $4318 Broadcom Corporation, BCM4318 [AirForce One 54g] 802.11g Wireless LAN Controller
 Vendor $106B Device $003F Apple Computer Inc., KeyLargo/Intrepid USB
 Vendor $1033 Device $0035 NEC Corporation, USB
 Vendor $1033 Device $0035 NEC Corporation, USB
 Vendor $1033 Device $00E0 NEC Corporation, USB 2.0
 Vendor $106B Device $003B Apple Computer Inc., UniNorth/Intrepid ATA/100
 Vendor $106B Device $0031 Apple Computer Inc., UniNorth 2 FireWire
 Vendor $106B Device $0032 Apple Computer Inc., UniNorth 2 GMAC (Sun GEM)
2·Ram Disk:T% lame_ppc AKsack.wav
LAME 3.98.4 32bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 16538 Hz - 17071 Hz
Encoding AKsack.wav to AKsack.wav.mp3
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
 10529/10529 (100%)|    [b]0:24[/b]/    [b]0:24[/b]|    [b]0:24[/b]/    [b]0:24[/b]|   11.086x|    0:00
-------------------------------------------------------------------------------
   kbps        MS  %     long switch short %
  128.0      100.0        76.1  13.4  10.5
Writing LAME Tag...done
ReplayGain: +0.5dB
2·Ram Disk:T% lame_vmx AKsack.wav
LAME 3.98.4 32bits (http://www.mp3dev.org/)
Using polyphase lowpass filter, transition band: 16538 Hz - 17071 Hz
Encoding AKsack.wav to AKsack.wav.mp3
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III (11x) 128 kbps qval=3
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
 10529/10529 (100%)|    [b]0:16[/b]/    [b]0:16[/b]|    [b]0:16[/b]/    [b]0:16[/b]|   16.362x|    0:00
-------------------------------------------------------------------------------
   kbps        MS  %     long switch short %
  128.0      100.0        76.1  13.4  10.5
Writing LAME Tag...done
ReplayGain: +0.5dB

/me pats the Mac mini G4 & PowerBook G4 ;-)

[Piru]

It appears mr Mufa claims that I've "faked" the results. Thus I've decided to release couple of photographs showing the results:

http://sintonen.fi/pics/bench_raw_macmini_g4_15.jpg
http://sintonen.fi/pics/bench_raw_powerbook_g4_167.jpg

Excuse the poor quality of these pictures. They should still serve their purpose however.

[Piru]

Please use Linux Mint 11 on both
http://mintppc.org/
Or Debian Wheezy on both and redo the math.

Unfortunately no-one is willing to run benchmarks on their X1000 anymore. I wonder why that is...

Looking forward to compare those tests to these and let us know of how much space there is for MOS and OS4 to grow on same hardware.

Actually MorphOS is faster than Linux in some areas, so I don't see much point in these tests.