Note: These observation relate to WSJTX 2.4.0-rc1. This is a release-candidate Beta-test version of the program, which will certainly go through many revisions before a final general release. Unless you are running WSJTX v2.4.0-rc1, observation made here may not apply to your version of WSJTX

UPDATE: The averaging scheme used in WSJTX 2.4.0 -rc3 and -rc4 is slightly different to the one described below. In the earliest releases of WSJTX 2.4.0 all transmissions since the last the average was cleared (either manually or by a decode) where given equal weight in the average. If many noise periods were received before a signal and were included in the average, this resulted in no decode being obtained, even if a set of 3 or 4 consecutive received signals could have been combined for a decode. This has now been changed to s scheme where the last 4 received signals are given equal weight in the average, and previous received signals are given exponentially less weight. In practice this turns out to be a better technique almost all the time.

Q65 Averaging test at very low SNR

Previous tests with simulated files have shown that averaging in Q65 can be very effective, even when it is observed that 20 to 30 signals must be averaged in order to obtain a decode (see Q65-300E averaging simulation tests). Averaging certainly works well and can significantly extend the decode sensitivity. In practice, during QSOs which require a small number of files to be averaged to achieve a decode (Q32 or Q33 decode codes), averaging in Q65 is a powerful tool of great practical use in enabling the decoding of weak signals. In these circumstances there's probably little that could be done to improve it and certain nothing that needs to be done. It works! Use it!

There are other (rare) circumstances in which one aspect of the averaging scheme may be an issue when averaging a large number of very weak signals in order to achieve a decode. This could be in a situation of deliberate extreme QRP operation, or, for example, on 48Gha and 76Ghz, where available equipment pretty much forces amateur operation into the "extreme QRP" category and multi-period averaging may be essential

The question under consideration here is whether the order in which the signals are received affects the average decoding. Or whether averaging a subset of signals can result in a decode when the use of the whole set does not. Since, in the current version of WSJTX 2.4.0 - dev (10f574), I believe all the signal FFTs are averaged (the symbol spectra are added), theoretical considerations might suggest that in some cases, adding another signal to the average could reduce the resulting averaged signal SNR as well as increase it. It might therefore, by selecting which signals were included in the average, be possible to obtain a decode form a subset of signals, when averaging the whole set did not produce a decode.

Simulated files could be used for this test (and indeed I have run the tests using simulated files), but the use of "off-air" recording ensures that any "real world" factors not present in the simulations are present in the data used for the test. In this case eight degraded "off-air" files were renamed as 0000, 0100, 0200, 0300, 0400, 0500, 0600 and 0700. For the Q65-30 modes, these files will all be seen by the WSJTX decodes as being in the "even" channel, so all 8 will average (rather than the averaging being odd/oven and so resulting in two sets of 4 transmissions being averaged).

With just two files, either of which will not single period decode, it doesn't matter what order they are in. With three files there's now a choice. You have (1,2,3), (1,3,2), (2,1, 3), (2,3,1), (3,2,1) and (3,1,2). If two of the files (say #2 and #3) are relatively strong and will just decode when averaged and the third (#1) is very weak (close to just noise) the (1,2,3) may not decode, while (3,2,1) may decode (after 2 files have been averaged).

What happens in practice

For this test I took 8 "off-air" transmissions from W2HRO using Q65-30B. These signals were quite strong, so all the signals were degraded by the addition of 8.8dB of noise (normal distribution), resulting in files that would individually not produce a single period decode, but could produce a decode when averaged. In order that all these 30B files would average in a single channel, they were renamed as 0000, 0100, 0200, 0300, 0400, 0500, 0600 and 0700. Decoding these files in sequence results in all 8 files being averaged (unless a decode occurs with less than 8 files). The files used may be downloaded here dg-8p8.zip

So what do we see when these files are run though WSJTX for decoding. Here's a list of a small number of runs with files in different orders. Here 0=0000, 1=0100 etc.

Run #1 Files in numerical order 0, 1, 2, 3, 4, 5, 6, 7 - result NO DECODE
Run #2 Files is random order 6,1,4,5,2,7,3,0 - result NO DECODE
Run #3 File is different random order 0,2,5,3,4,7,6 - DECODE as Q37
Run #4 files in different random order 6,3,4 - DECODE as Q33
Run #5 Files in different random order 6, 4 - DECODE as Q32

I'm sure there are other sequences which do and do not decode, but these few examples illustrate the point well enough.

The conclusion here is clear. You can get a decode with (the right) 2 files or you can fail to get a decode averaging all 8 files. That selection of which files are averaged can significantly affect whether of not a decode is achieved. I will again note that these are "off-air" files. All that has been done to then is that they have been equally degraded (in this case by 8.8dB) so that they represent files with a signal strength low enough to require averaging. None will decode in a single period. This is not a surprise, in fact it would be expected behavior in a system where all files were averaged and not all files were of equal strength.

What does this mean?

First what does it NOT mean. It does not mean that WSJTX Q65 averaging does not work. It quite clearly does work and works very well, especially for signals that are quite close to the single period threshold. If two files average to produce a decode, there is no influence of file choice or file order on that decoding probability. The same is probably true most of the time when 3 files are needed (though in this case it is possible that given a different file order, a decode might have been possible with just two of the files). As the number of files being averaged increases, there are more possibilities of the "default" file order (as received) not being the best possible combination and/or ordering of files to achieve a decode. So unless you are not seeing an average decode after 3 or 4 transmissions, the effect described here probably isn't really affecting your ability to get an average decode in a reasonable time.

What can you do about this?

This first assumes you think there's a need to do something, but if you do, I'll leave this one to the mathematicians! Trying every possible set of files when there are a large number of files by a "brute force" algorithm isn't very practical for an on-air QSO exchange. It would take too long. Post processing the files you could use a brute force algorithm and try all possible combinations (or is that permutations..?) but you probably rapidly run into a time constraint as the number of files increases. The computational time required to find the first decode (assuming there is one) will not scale linearly with the number of files. Without a "smart" algorithm it could (unless you were very lucky) take unreasonably long to sort through large numbers of files. Of course in post-processing, you aren't looking for the best solution here. If you have 25 files you don't need to know the minimum number of files (and which files they are) which will result in a decode. You just want to find one combination which produces a decode (assuming there is such a combination). Mathematicians probably have techniques for dealing with such problems, but I don't! I believe that for 25 files there are 15,511,210,043,330,985,984,000,000 possible different orders in which they could be run!

You can't just pick "the best files", because it may not be possible to determine which are the best files without actually doing the averaging. Again, this is something for the mathematicians to answer. Perhaps there is some property of the signal FFT which may hint at a measure of whether it's a "good" or "poor" candidate for averaging. That's beyond my skill level. A quantum computer which looked at all possible combination at the same time would be nice, though such a machine doesn't yet seem to be commercially available!

A simple algorithm to find the "good" files may be possible, though it doesn't speed up a decode! The last file in any set of files which produces a decode must be a "good" file, since when it is included as the final file in an average, a decode occurs. This file could them be moved to the first file of all subsequent sets and the process repeated, again moving the last file of an average decode to the front of the queue. If you keep repeating this eventually you will get close to a set of "good" files and finding the minimum number of files required for a decode. The problem is that you need to get a decode to start this, and since your goal is to get a decode, you've solved the problem at that point. Finding the strongest set of files may be interesting, but isn't required to make a QSO.

If you were running 300s periods on alternate periods, you would have 5 minutes in which to look for a decode of any previous set of files. With one file in the average, there's nothing to do. With two files in the average, their order doesn't matter. If you are going to get a decode, you will get a decode with either order. With three files in the average there are now 3! = 6 permutations and you could run them all in less than a minute or so. With 4 files things start to get slow since you have 4! = 24 different permutations of 4 files to run. That would take a few minutes. With 5 files you have 5! = 120 permutations of 5 files to run which might take 10-15 minutes on a typical PC. !ith 6 files you have 6! = 720 permutations of 6 files to run which is going to take over an hour, and with 7 files the time extends to 8 or 9 hours. Of course, on averge, if there is a set of files which will decode you may not have to wait the full time to see a decode, but to be sure there isn't one, you do.

Bottom line - remember to clear and reset averaging when appropriate

For the vast majority of situation where signals don't quite decode in a single period, averaging works very well when used in the proper manner.

However it's very important to make sure that you clear the average right before you start a QSO attempt using averaging. If the system has been listening to (and accumulating) noise for a while and then you start a QSO, all that noise will be included in the average before any weak signals are added. This will lower the likelihood of an averaged decode. It would probably be a reasonable idea for the "Clear Average" button to light up in RED at any time when there is an entry in the average accumulator. If averaging was ON, but there were no files stored yet, it could be GREEEN, and if averaging were OFF, the button would be the normal grey default button color. This could minimize user errors. It's easy to forget to clear it before a QSO (after a lot of average testing I have personal knowledge of this!). The average will clear automatically if "clear after decode" is selected of course, but since the system doesn't know when you start listening to a weak signal, it can't do the reset for you. If you move from a weak signal to another weak signal, you have to remember to clear the average before starting to accumulate data from the new signal.