Schedchk

Schedchk - Check and fix the Humax recording schedule

Introduction

While it is "only telly" it can be extremely annoying when we find that a programme we had scheduled to record fails to record correctly. Some failures can be attributed to user errors but a considerable numbers of problems are attributable to the broadcasters and problems in the Humax software not anticipating the ever more creative ways the broadcasters find to make life complicated!

The origins of schedchk go back several years to a forum postwhere I proposed the idea of automatically detecting when programmes were delayed or moved to another channel (e.g. BBC One -> BBC Two) because of overrunning sporting events or breaking news events. Nothing much happened at that time since Wimbledon only last two weeks and other last minute programme changes are by their nature unpredictable making testing difficult.

However now we live in Interesting Times and the unpredictable has become routine, every day the BBC1 showing of Pointless is moved to BBC2 to make way for the Corona Virus Daily Update. One might think that after this long the BBC would take the hint and schedule Pointless on BBC2 but no they persist in scheduling it on BBC1 and only update the schedules 3-72 hours in advance! Since Pointless is one of our favourite shows I was provoked into resurrecting the Schedchk idea and finally it is now ready for a wider audience though there may still be a few rough edges.

The program relies on the EPG being kept up-to-date to allow it to find alternative showings of programmes and, especially with last minute changes, that can't be relied upon and sometimes there are no alternate showings so it will not be infallible but hopefully it will help you avoid some missed recordings,

To allow schedchk to promptly update the recording schedule it is important that Real-time scheduling is enabled via the Advanced Settings section of the Settings page

I have found several different categories of potential errors in the schedule and these are listed below with examples of how they can occur and action schedchk takes when attempting to fix them.

Schedule Check Categories

EPG changes

The most basic form of schedule change is when the broadcaster changes the start time, duration or scheduled episode (event-CRID change). Usually the Humax is quite good at spotting these changes but sometimes it doesn't immediately.

Schedchk will resolve this type of error by updating the schedule entry with the new details.

Checking for this type of problem can be by-passed by specifying the -noEPGchg option in settings or on the command line

Missing EPG entry

Sometimes the event-id associated with a programme changes and the Humax is not good at resolving these, on occasion it can get very confused resulting in it scheduling two episodes of the same series at the same time!

Changes to the event-id can occur when a programme is moved another channel (e.g. Pointless moving from BBC1→2), when a programme is added to the schedule (ITV assigned new event-ids to all programmes following the Queen's recent broadcasts), a different episode is scheduled (e.g, the duplicate example above) or if a programme is cancelled and replaced by something else.

Schedchk attempts to resolve this type of error by searching for another showing of the same programme (by event-CRID) elsewhere in the EPG and if another showing (that doesn't cause a conflict) can be found, it will schedule that showing instead. If there are no matches by event-CRID it will look for a programme of the same name scheduled for the same time in case the Broadcaster has changed event-id, event-CRID and series-CRID and scheduled an episode from a different series of the programme. If duplicate episodes occur it will delete the episode that no longer appears in the EPG.

Checking for this type of problem can be by-passed by specifying the -noMissEPG option in settings or on the command line

Conflict resolution

Conflict arise when three of more programmes are scheduled to be recorded at the same time (Two or more on HD model). This can arise if a user ignores conflict warnings when scheduling, when a news day's EPG data is loaded, or if the EPG changes.

Schedchk attempts to resolve this type of error by searching for another showing of the same programme (by event-CRID) elsewhere in the EPG and if another showing (that doesn't cause a conflict) can be found, it will schedule that showing instead. When considering alternatives schedchk will attempt to find a match on a channel with the same definition (HD or SD) as the original but if there is no match it will consider matches on channels with a different definition.

Schedchk is quite simple minded in its approach to conflict resolution and will not always choose the best solution for your viewing needs. For complex conflicts or where you want to select which programmes you don't want delayed you should attempt to manually resolve conflicts using the Visual Schedule displays in the webif or RS before schedchk attempts automatic resolution.

At the moment Split recordings are not considered for conflict resolution.

Checking for this type of problem can be by-passed by specifying the -noConflict option in settings or on the command line

Unscheduled episodes

The Humax normally manges to automatically schedule new episodes of a series but can sometimes miss changes that happen in the middle of the scheduled sequence

Schedchk checks the EPG by series-CRID and adds any missing

Checking for this type of problem can be by-passed by specifying the -noUnsched option in settings or on the command line

series-CRID changes

Often broadcasters change the series-CRID of a long running programme causing the next episodes of the programme to be not recorded. Sometimes this is because it has switched to another series of the programme (as if we really care whether it is repeats from series 51 or 55) but other times there is no obvious reason for the change.

Schedchk handles this by looking by programme name for a new series with matching name in the same time slot on the same channel and scheduling a new series recording for the programme.

Prefixes such as New: are ignored but the checking can miss some programmes and find false matches in other cases:

• Variations in title, Pointless would match Pointless Celebrities but not vice versa
• Programmes with variable time slots such as the F1 Grand Prix races
• Programmes moved by conflict resolution, If, for example, an series episode has been moved to a +1 channel the series-CRID check is liable to consider the +1 series a new unscheduled series

Checking for this type of problem can be by-passed by specifying the -noSCRID option in settings or on the command line

One-off episodes of a series

Sometimes broadcasters schedule episodes of a series as 1-off programmes without a series-CRID. Often this is deliberate when filling in a gap in the schedule but other times it is the result of broadcaster incompetence! Recently, Pick broadcast several episodes of the Sherlock series as 1-off programmes 😥

Schedchk handles this by looking by programme name for a 1-off programme with matching name on the same channel and scheduling a recording for the programme.

Prefixes such as New: are ignored but the checking can miss some programmes and find false matches in other cases:

• Variations in title, Pointless would match Pointless Celebrities but not vice versa
• Programmes moved by conflict resolution, If, for example, an series episode has been moved to a +1 channel the 1-off check is liable to search the +1 channel

Checking for this type of problem can be by-passed by specifying the -no1off option in settings or on the command line

Not Recording

Occasionally a programme fails to start recording. This can be due to an unresolved conflict, overrunning programmes, last minute schedule changes etc. but some failures are unexplained.

When a programme is due to be recording schedchk checks whether there is a file for the recording and whether it is growing in size. If the programme is not recording it searches by event-CRID for an alternative showing and, if a suitable alternate is found, schedules it as a 1-off recording.

Since some late starting is normal the checking starts a short while after the scheduled start. By default this is 300 seconds (5 minutes) but this can be modified by specifying the -recWait option in settings or on the command line

Checking for this type of problem can be by-passed by specifying the -noNotRec option in settings or on the command line

Note: Since active recording schedule entries are deleted on Humax start-up schedchk can't schedule alternates for recordings truncated by a power failure or Humax crash 😥.

Split Recordings

Split recordings are where a long programme such as a film has a shorter programme, often a news bulletin, in the middle of the longer programme. While they may appear to be a 2-episode series they are not treated as such by the broadcasters or the Humax and have their own (very badly designed) set of rules which the Humax only partially implements. Split recordings contain #number at the end of their event-CRID (eg /X4454865#01) and both halves of the split recording are required to have the same number, known as an IMI (Instance Metadata ID). The problem arises when the same film is repeated later in the week with exactly the same event-CRID and IMI, under the rules if there is more than three hours between them they should be treated as different programmes but the Humax treats them all as additional parts of the same programme.

This is a particular problem on channels such as Sony Movies which split most films into 2 parts and regularly repeat films up to three times a week. So you can attempt to schedule just one showing of a film but the Humax will schedule all six parts and some of the unwanted recordings can cause schedule conflicts.

Schedchk handles split recordings by leaving those with one or two parts schedule alone, if there are more than two parts scheduled it checks whether there is more than a 3-hour gap and whether there are any conflicts. The first, non-conflicting, showing is kept with a dummy IMI to prevent the Humax rescheduling and the other showings are removed from the schedule.

Checking for this type of problem can be by-passed by specifying the -noSplit option in settings or on the command line

Other controls

Threshold

The Threshold setting (-thresh) controls how far into the schedule schedchk should look for problems. The default, unless changed in settings, is 36 hours which seems adequate for resolving potential problems.

You may wish to increase it on infrequently used machines.

No Auto

Normally schedchk runs as part of the regular Auto processing. If you would prefer to run it on ad-hoc basis from a command line or set up your own schedule with crontabs you can specify -noAuto in the Schedchk section of the settings page.

No Alerts

Schedchk generates webif warnings when it makes changes to the recording schedule to let you know what has happened.

Alert messages

If you would prefer not to receive these pop-up messages you can specify the -noAlert option in settings or on the command line. Messages (whether or not alerts are enabled) can be viewed in the schedchk.log file using the WebIF log file viewer WebIF >> Diagnostics >> View Log Files, or in auto.log if schedchk is run as an auto-process.

Note: The example shows that sometimes broadcasters makes changes and then reverse them in short order.

Test Mode

If you specify the -test option no schedule changes will actually be made

In-place updating

If you use the visual schedule webif/RS pages to "Record this showing instead" it will generate a skip entry for the current episode and a one-off recording entry for the replacement episode. Schedchk uses a different approach and where possible updates the existing series entry with the new episode details.

The main advantages of this approach is that the recording will automatically go into the series folder and any folder related webif settings (auto-decrypt, shrink, detectads etc) will be applied as usual. You will only have one entry in the recording schedule rather than many and you won't have to move recordings from the My Videos folder.

It can look a bit odd to have episodes from different channels in the same series but both webif and RS handle this correctly

Which of the channels will show as the 'main' channel for the series depends on the channel of the next scheduled episode

Minor disadvantages are:

• The Humax won't add new episodes while the 'wrong' channel is main (but Schedchk 'Unscheduled episodes' will add them instead)
• Schedchk 'SCRID change' and '1-off episodes' will check the alternate channel when it is main (more work needs to be done in this area)

If you would prefer more traditional skip and 1-off scheduling you can specify the -noInplace option in settings or on the command line.

Progress Messages

The -d option controls the level of messages produced:

• 0 - Schedule changes and errors only (also displayed as webif notifications)
• 1 - Intermediate level
• 2 - Detailed progress messages

Messages are sent to a log file schedchk.log, which can be viewed via WebIF >> Diagnostics >> View Log Files, or auto.log if schedchk is run as an auto-process.

Schedchk settings

Initially all schedchk processing options are enabled but the default settings can be overridden on the webif Settings page, any changes made here become the new defaults for schedchk.

More detail on each of the options is given in the sections above.

SchedChk Settings Page

Schedchk command line

As well as running automatically every few minutes as part of the regular Auto process you can also run schedchk on demand from a command line. Options specified on the command line override values set on the settings page for this invocation of schedchk only, they don't change the default values.

schedchk = Check for recording schedule issues and attempt to fix them

schedchk -h            = produce this help
schedchk -rsv          = scan the recording schedule

Options      Default (unless changed in Settings)
-thresh      36   = Don't check for problems more than x hrs away from start
-d           0    = Produce more detailed output
-test        n    = Don't make updates
-noAuto      n    = Don't run regularly as part of Auto-Processing
-noAlert     n    = Don't generate webif alert messages for changes made
-noConflict  n    = Don't check for conflicts
-noUnsched   n    = Don't check for episodes missing from schedule
-noSCRID     n    = Don't check for series CRID changes
-no1off      n    = Don't check for episodes scheduled as one off programmes
-noNotRec    n    = Don't check for failures to record
-recWait     300  = Don't assume recording failed until x seconds late starting
-noMissEPG   n    = Don't resolve missing epg entries
-noEPGchg    n    = Don't resolve epg time/ duration/ event crid changes
-noSplit     n    = Don't resolve repeated split recordings
-noInplace   n    = Update reservation by skipping & 1-off rather than inplace

Note: Options are not case sensitive

-noAuto has no effect when specified as a command option, it needs to be set via the settings page.

Running via crontabs

As an alternative to running schedchk as part of the standard customised firmware Auto processing it also possible to schedule schedchk to run at different intervals using the crontabs file.

You might want to do this if:

• You normally disable Auto processing during recordings but would still like to take advantage of Not Recording checks
• You want some checks made at a different frequency from others
• You want process messages written to a dedicated log file

You can edit the crontabs file via webif→Diag→File Editor→Commonly edited files→/mod/var/spool/cron/crontabs/root

Editing crontabs file

In the example above schedchk will run a Not Recording check every 4 minutes and check for SCRID changes and 1-off recordings every 4 hours at 17 minutes past the hour, output will be written to schedchk.log

Guides to crontabs time formats are available online

fmtrsv - Format a rsv.db table entry

Fmtrsv is a simple command line utility created to help with displaying the contents of the rsv.db where schedule entries are stored. It is included with the schedchk package to help in gathering information for debugging purposes. Entries displayed can be filtered with a number of options

# fmtrsv  -h
fmtrsv = Formats schedule entries from the rsv.db

fmtrsv options
no options will list all reservations

Options
-h                  produce this help
-n,-name    xxx   = select recording name contains
-s,-slot     n    = select slot
-l,-lcn      n    = select channel number
-c,-cname   xxx   = select channel name contains

Sample output

# fmtrsv -s 16
ulslot               = 16
ersvtype             = 3              Recording
hsvc                 = 131097
nsttime              = 1590429600     25/05/2020 19:00
szsttime             = 00000000000000
nduration            = 3600           01:00
erepeat              = 0              no repeats
usevtid              = 12270
szevtname            = i7Code-Breakers: Bletchley Park's...   (Code-Breakers: Bletchley Park's...)
ulPreOffset          = 0
ulPostOffset         = 0
ulProgramId          = 0
ulSeriesId           = 0
ucVolume             = 0
ucInputMode          = 0
usChNum              = 0
ucRecKind            = 4              series
ucCRIDType           = 50             series
szCRID               = FP.BBC.CO.UK/B-3P5L5
szFPBRecPath         = Code-Breakers: Bletchley Park's...
szRecordedProgCrid   = 1FP.BBC.CO.UK/B/KZSO7|
szEventToRecord      = 1FP.BBC.CO.UK/B/KOEPL|
aulEventToRecordInfo = {131097 1590429600 1590433200 12270}
bRecomRsv            = 0
usLastRecordedEvtId  = 0
eReady               = 30
szSvcName            = BBC TWO
usLcn                = 2
  Svcid  Start                         End                             Evtid  CRID
 131097  1590429600 25/05/2020 19:00   1590433200 25/05/2020 20:00     12270  1FP.BBC.CO.UK/B/KOEPL
Skiplist Svcid:Evtid
131097:12390
========================================================

Other Stuff

Change history

Date	Version	Author	Description
2020-06-05	schedchk 0.1.0	mymsman	Intial Version All new!
2020-06-15	schedchk 0.1.1	mymsman	Minor fixes Fixed problem with scrid change that stopped change to Cardinal series crid being recognized Check for gaps in EPG and don't reschedule programmes already scheduled in gap.

Acknowledgements

af123

For providing the cf framework that underlies everything and who's code I have copied and hacked.

/df

For investigating the problems with Split recordings and writing the rsv set method

everybody else

Who has assisted through the forums

Future Enhancements

• Detect and reschedule recordings active at time of crash
• Improve Series CRID change detection to reduce false positives