Latest revision as of 18:25, 18 January 2019

DetectAds - Eliminate watching the commercial breaks!

Overview

If you are fed up with watching all of commercials for this, that, and the other or you are an habitual user of the fast forward/skip buttons on the remote when an ad break starts then DetectAds is the package for you!

DetectAds analyses a recording to detect the commercial breaks and places bookmarks at the start (optional) and end of each break so that you can just use the Bookmark button (arrowed in picture) to jump instantly to the next bookmark. Alternatively you can leave the remote languishing between the sofa cushions and have DetectAds physically remove the commercials and EPG data from the recording shrinking the recording size considerably.

DetectAds can also generate a thumbnail image for the programme for display in the on TV media menu.

Processing modes

DetectAds can be used in many different ways and which is best for you will depend on your preferences and viewing habits.

There are also a number of Processing Options which can be used to tailor the processing.

To crop or not to crop - that is the question

Bookmarks leave the original recording intact but allow you to use the Bookmark skip button on the remote control (arrowed in picture)to quickly jump to the end of the ad break. Cropping creates a copy of the recording with the ad breaks physically removed eliminating the need to use the remote to skip ads.

Detecting ads can never be 100% accurate with the risk of it failing to spot an ad break, incorrectly identifying part of a program as an ad break or mis-positioning the end of break markers.

In several years of using DetectAds I have seen very few detection errors with just a few failures to spot an ad break so I am happy to allow cropping to take place automatically - the original can always be rescued from the dustbin if needed (I do recommend use of the Undelete package for safety)

However everybody's viewing habits differs and the success of DetectAds may vary by channel or program time so it is probably worth not using the "Delete input" option until you are comfortable that cropping is working reliably for you. If need be you can specify which Processing Options are used for different series.

If you do not use DetectAds to crop programmes the Bookmarks set by DetectAds can be used to crop the programme a a later time. Note: Do not use the "Only Bookmark ends of ad breaks?" option if you want to retain the ability to run a manual crop,

You may notice a short period of visual pixelation and/or sound drop-outs at crop points.

Which programmes to run DetectAds on

You can select which programmes to be processed by:

Channel, there is a Channel exclusion list.

By default this include BBC* to exclude the main BBC channels but you can add to the list if there are other channels you wish to exclude from processing. Radio programmes are always excluded.

By series, you can set Folder Flags to include or exclude specific series folders from processing or to specify different Processing Optionsfor a series.
By Sweeper, you can also set up Sweeper rules to select which programmes are to be selected for processing and which Processing Options are to be used
On demand, DetectAds can also be run on specific programmes from the Browse or FlexView menus, advanced users could also invoke it from a command line or script file

When to run DetectAds

During recording, Known as Chaserun mode this decrypts and detects ad whilst recording is in progress so the processing completes within a few seconds of the recording finishing. You can even,with some limitations, start to watch your program ad-free whilst it is still recording. This mode works well for many users but it is possible that the extra processing load could impact the recordings in progress.
After recording, When a recording completes the Auto process checks what post recording processing is required. In Traditional processing mode it runs after Auto decryption has completed.
Deferred processing, to eliminate the possibility of interference with recordings it is possible to defer DetectAds processing to a quiet time (possibly overnight) when no recordings are in progress - you can use a repeating reminder o ensure the box is switched on at the designated time.

Installation

DetectAds is in the Packages catalogue. Chaseget, ir and some other packages used by DetectAds will be installed/upgraded automatically as needed.

In common with all processes that manipulate the recording file, DetectAds needs to work on a decrypted file or to decrypt the file itself. For this, the minimum requirement is to ensure Content Sharing is enabled via Menu >> Settings >> System >> Internet Setting >> Content Share = On. This setting can be enforced using the bootsettings package.

For more information regarding the need for, and implementation of, decryption see Encryption or Decryption Guide (hummy.tv forum).

Automatic Processing

Traditionally automatic webif packages have been run sequentially on completed recordings under the control of the "auto" process which ran at fixed intervals. DetectAds can be run in this mode triggered from auto when auto-decryption completes.

Recent enhancements have improved the speed of this post-processing by triggering auto when a recording ends and allowing decryption to start before the system has DLNA indexed the recording but it still takes a significant time to sequentially run a number of processes on a recording.

DetectAds is an exceptionally slow process taking about 15 minutes to process an hour of SD recording which means that with the time to decrypt recording and perform crop there is a considerable delay before a fully processed recording is available for viewing.

DetectAds now introduces as an option a radically different processing model, with "Detect whilst recording" (also known as Chaserun) the detection process runs in parallel with the actual recording so the processing completes within a few seconds of the recording finishing. You can even,with some limitations, start to watch your program ad-free whilst it is still recording.

With both processing modes you can choose to exclude certain commercial free channels such as BBC and Radio stations from processing (Ads on radio stations are not recognized by DetectAds)

Some users have reported that when the system is exceptionally busy such as when recording multiple HD channels and playing other recordings that running DetectAds can cause Picture and Sound breakups on the current recordings, if you experience such problems you can limit the amount of CPU used by the processor intensive part of DetectAds or / and defer processing until the system is quieter.

Detect whilst recording -Chaserun processing

Ad detection requires a decrypted recording to process so when detecting while recording is in progress there can be three copies of the recording in existence simultaneously: The encrypted input file, the decrypted intermediate file and the cropped & shrunk output file.

To disambiguate between the various files in both the webif and the standard Humax TV interface DetectAds adds the qualifier -dec to the file name and the word Decrypt to the programme title for the decrypted file and adds the qualifier -crop to the file name and the word Crop to the program title for the cropped file.

At the end of recording the input files will normally be deleted and the output renamed to the original file name and title. File renaming will not take place if any of the following is true:

The "Dustbin/delete original recording after processing?" option is set to NO
The Input or Output files are Inuse (e.g. being played or processed by another task)
There is a length error detected (see below)

There is a small chance that there will be a failure in the retrieval of the decrypted file resulting in a file that is longer or shorter than it should be. This seems to occur if the system is DLNA indexing the recording whilst it is being retrieved. Since retrieval is usually within 30 seconds of actual recording it is a rare event and usually only affects the closing credits/ trailing ads. If it does occur the input recording will be left in situ and not replaced by the -dec (which will be deleted) or -crop files (which will be kept, If the length error is significant, more than 30 seconds long, the input file will be automatically queued to retry ad detection since it is unlikely to recur again.

It is also possible that a poor quality recording due to low strength or transmission glitches will cause the detection process to drop frames resulting in an apparently shorter recording leading to misplaced bookmarks and cropping.

If either form of length error is detected a system notification message is generated to warn you that bookmarks and cropping maybe unreliable, the input file is not deleted and -Len err is appended to the title of the -crop file (if auto cropping is active)

The DLNA server is not active in half-awake recording state so ChaseGet uses the IR package to force the system fully awake if needed. After the recording ChaseGet will attempt to turn power off when it it safe to do so but it is not easy to determine if the user has started to watch the TV in the mean time without pressing any buttons on the remote.

To reduce the risk of upsetting important family members when the box shuts down without warning (The Customised Firmware has no ability to put a 2 minute standby warning on screen like the Humax does) you can specify a time range in the ChaseGet Settings during which automatic return to standby won't operate.

You can also choose in the settings whether to Mute the sound when ChaseGet does wake the system from standby outside the peak time-range. This would prevent a sudden blast of sound if the system wakes up in the middle of the night for a scheduled recording and the TV has been left on or auto-switches on HDMI activity. A user un-muting the sound also indicates that someone is watching the TV and so ChaseGet would not then return the system to standby.

ChaseGet will return the system to StandBy if during scheduled checks which run every 20 minute all of the following are true.

The system was awakened from Standby by ChaseGet
The time is not within the exclusion time range
The system isn't currently in standby
No significant programs are running (including auto, detectads etc)- full list is in /mod/etc/chaseget.conf and can be extended if needed
There are no active PuTTY / Telnet psuedo-terminals
The system hasn't been been rebooted since wakeup
No recording is being played (on set or via remote access)
There is no active Reminder event
No remote control buttons have been pressed since wakeup

Traditional processing

Unlike the chaserun processing there is a prerequisite for the file to decrypted first and bookmarks are added to it so there is no output file unless the Crop option is selected, for consistency with chaserun processing deletion of the input depends on the 'Delete input' option. If the input file is not deleted then the output file name is suffixed with -crop.

It is possible that a poor quality recording due to low strength or transmission glitches will cause the detection process to drop frames resulting in an apparently shorter recording leading to misplaced bookmarks. If a significant length error (>5 seconds) is detected a system notification message is generated to warn you that bookmarks maybe unreliable, the recording will not be cropped automatically.

Setting the folder auto-decrypt option is required for the DetectAds process to be trigged but if you are using Flatten or Sweeper to rename/move files between folders it is recommended that the auto-decrypt option only be specified on the target folder to prevent the chance of DetectAds not finding the file because it has been moved.

If you wish wish to defer DetectAds processing to a time when the system is less busy you can specify a target start time on the Setiings page, the target time is the earliest start time but processing may start later depending on when the regular DetectAds task starts and how many other items need to be processed on the queue. You must of course ensure the the system is switched on at the target processing time and remains on long enough to process all of the queued items (possibly set up a scheduled repeating reminder event)

Folder Flags

There are two new folder flags Auto DetectAds and No DetectAds

These work in conjunction with the existing Channel exclusions rules so it is not necessary to add the flag to each directory if you use the existing DetctAds Auto processing modes (Automatically process whilst recording in progress? (chaserun) or Automatically process recording following auto-decryption? (traditional)

You can also use the Auto DetectAds flag to process existing recordings in a folder and to change the Processing Options

The Auto DetectAds flag means all recordings in a folder will be processed regardless of the recording channel and whether they have been decrypted. The No DetectAds flag means none of the recordings in a folder will be processed regardless of the recording channel. In the absence of a flag recordings will be processed during recording (chaserun) or following decryption (traditional) if they are on a channel that has not been excluded by the channel filters.

If you routinely want to detect ads in all commercial programmes, as I do, you don't need to use the new flags at all. If you only want a subset of programmes processed or to change Processing Options you can use the flags to include/exclude directories overriding the channel filters

For instance you could set No DetectAds on the 'My Videos' root folder to prevent auto ad detection of one-off recordings (I don't recommend setting Auto DetectAds on 'My Videos' since this would override the channel selection filters and cause detection of all one-off recordings including those on BBC channels)

To specify specific Processing Options for recordings in a folder select "Enable Auto DetectAds" and then "Edit Auto DetectAds options" on the folder Opt+ menu, this will bring up the file editor where you can specify the required options on the first line of the file. The available options are presented as an aide-mémoire.

Folder specific options file

The processing is (deliberately) different from the other auto options where you have to have a flag set before any processing happens.

Sweeper

Another method of automating DetectAds processing is to set up Sweeper rules to select which recording should be processed by DetectAds. This allows you much finer control of the process than the other automation methods that only allow for exclusion based on channel name or folder. Since Sweeper only processes files once they have completed recording and are not in-use it is inevitably slower than using Detect whilst recording.

When the DetectAds package is installed several new options are added to the Sweeper rules menus. These allow you to:

Check whether the recording has been flagged as 'Ad-Detection done'
Check whether the recording has already been queued for detection
Add the recording to the auto processing queue, optionally specifying the time for processing to start,

To assist with setting up the rules a new sample set has been added to the pre-defined ruleset list. These are shown and explained below.

Sweeper DetectAds rules set

There is one rule in the set.

The rule may need to be modified to meet your own requirements, as supplied it is a file rule (recursive into child folders) that adds to the auto processing queue recordings that, have no bookmarks, have no Addetection flag set, are not on a BBC channel, and are not already on the queue.

If you have other rules that rename or move the recording file they should precede this rule.

You can specify DetectAds Processing Options in the options field of the rule action

Specifying DetectAds options in Sweeper rule

Note: Previous versions of DetectAds provided specific DetectAds queuing actions and conditions which have now been superseded by the standard 'Queue recording for' action, the older actions remain for compatibly with pre-existing rule sets.

Invoking from Browse or FlexView

The Addetection icon is now displayed with the other recording flags in the file list for rcordings that have been processed by DetectAds.

Queuing multiple recordings

To process multiple recordings you can select the recordings, choose 'Ad Detection' from the list and then click on the 'Queue For' button at the bottom of the page. Remember Ad Detection is slow so be be sparing when adding batches of existing recordings to the queue. You can not specify Processing Options directly using this method - they are taken from the Settings page

DetectAds Opt+ panel

To process a single recording you can select "Detect Adverts" from the Opt+ pull down menu on the webif Browse file list. This will lead you to the main panel where there are several processing options:

DetectAds Opt+ menu panel

Run Analysis now: This processes the file immediately with the results displayed in the processing routine but due to the length of time taken to analyse a recording it is recommended that this option not be used unless you have a lot of spare time or need to investigate a problem!
Run Analysis in background: This option adds the recording to the Auto processing queue and returns. The queue is processed sequentially and, if the system is restarted, resumes on restart so if you have a number of recordings to be processed they can be safely added to the queue without waiting for preceding files to finish processing.
If you wish wish to defer DetectAds processing to a time when the system is less busy you can specify a target start time or ASAP on the panel. The target time is the earliest start time but processing may start later depending on when the regular auto processing task starts and how many other items need to be processed on the queue. You must of course ensure the the system is switched on at the target processing time and remains on long enough to process all of the queued items (possibly set up a scheduled repeating reminder event)

If the selected recording is already on the queue the two Run buttons are replaced by:

Remove from background queue: This removes the entry from the processing queue but it does not cancel processing for the recording if it has already started.
Clear Detection flags and Bookmarks: The resets the recording to allow reprocessing if you want to try different Processing Options

The Process Options field can be used to specify Processing Options overriding those specified on the Settings page. If the file has already been decrypted the program will be processed in 'traditional' mode otherwise it will be processed in the new 'chaserun' mode.

Command Line

DetectAds can also be started from a Telnet command line or from another script which could be useful if you have a batch of existing recordings to be processed.

Humax# /mod/webif/plugin/detectads/detectads -h
detectads = Detect ads in recordings and bookmark or crop them

detectads -h            = produce this help
detectads -aq recording ?options? = add recording to request queue
detectads -qq recording = query if recording on request queue
detectads -dq recording = delete recording from request queue
detectads -pr recording ?options? = process recording now
detectads -rs recording = reset ad-detection flag and bookmarks

All DetectAds options can be specified via the command line, see below, to change the default value use the Settings page to set processing options.

If the file has already been decrypted the program will be processed in 'traditional' mode otherwise it will be processed in the 'chaserun' mode.

Options

There are a considerable number of options that can be specified to modify the default processing of DetectAds. Many of these options are somewhat esoteric and should not normally be touched unless needed but others (shown in bold below) may be changed as required.

Some have a dedicated field on the Settings page but they can all be specified in the Other options field of Settings page, Process options field of Browse DetectAds page, Folder options file, Sweeper options field, command line.

If duplicate options are specified the last found takes effect, this means that options specified on browse panel, folder options, sweeper options or command line, override the settings page which override the built-in defaults.

The full option list is:

Option	Default (unless changed in Settings)	Description
-d -debug	n	produce detailed debug
-t	0	-aq only, target start time hh:mm or linux time value, 0 ASAP To defer processing until a time when little else is happening
-force	n	-aq only, don't check if DetectAds flag set Force reprocessing of a recording
-crop	n	Crop ad breaks from recording Can be specified as "Crop recording following ad detection?" on the Settings page
-bmEnd	n	Only bookmark end of ad breaks This can be used in conjunction with -crop but will prevent manual cropping Can be specified as "Only Bookmark ends of ad breaks?" on the Settings page
-bmSil	n	Bookmark all Silences (for testing) This sets a bookmark at every detected silence, it is only of use if you are experiencing mis-detection of ad breaks or are experimenting with the detection settings and want to see where silences are being found. NB: There is a limit of 32 bookmarks so it may not be possible to fully mark long programmes.
-delOrig	n	Delete input file after cropping Can be specified as "Delete (move to bin) original recording after processing?" on the Settings page
-tgt	=	Target path for cropped output, = means same dir as original Can be specified as "Write output recording to folder" on the Settings page
-cpuLimit	0	Limit ffmpeg cpu to n%, 0 no limit, requires cpulimit package,br> Can be specified as "Limit cpu usage to" on the Settings page
-misRdSec	30	Retry if more than n secs data missing during decryption Errors during decryption tend to be at the end of a recording and can often be recovered by retrying decryption once the recording is complete
-misDetSec	5	Warn if more than n secs data missing during ad detection Errors during the ad-detection phase are not usually recoverable and can result in misplaced (too early) crop points/ bookmarks, A warning is included in the cropped programme title and the original recording will not be deleted
-misDetAlarm	10	Webif notification if more than n secs data missing during ad detection In addition to the warning in the programme title a webif notification message will be displayed System Notification box If you don't want notification messages set this option to 999999
-delSec	180	Delay start of chaserun processing until n seconds after recording begins Initial delay to provide a reasonable batch of date to be processed
-retMax	2	Maximum retry attempts if decryption fails
-silDb	-70	How quiet a silence must be to be detected
-silMinSec	0.2	Minimum length a silence must be to be detected in seconds
-brkMinCt	5	Minimum number of silences in ad break Effectively the minimum number of ads (plus one) required in a commercial break.
-brkMinSec	60	Minimum length of an ad break in seconds Commercial breaks shorter than this wont be detected
-adMaxSec	65	Maximum length of a single ad in seconds Some daytime programmes have longer (90 second) charity appeals in commercial breaks so may need a 95 second value specified
-padSec	0	Adjust ends of ad breaks by n seconds If you experience sound or vision drop outs around crop points you can move the crop points by adding padding to reduce the impact of the drop outs on the actual programme content. Alternatively a negative pad value can be used if you see too much residual material from the breaks.
-makeThm	n	Make thumbnail if it doesn't already exist Creates a thumbnail image for the programme
-thmOffset	0	Thumbnail offset from programme start How many seconds after (before if negative) the detected start of programme should the thumbnail image be generated?

Settings

DetectAds Settings page

Options specified via the webif Settings section for DetectAds are:

Automatically process recordings whilst recording in progress? (chaserun): Select the new 'chaserun' processing mode, faster
Automatically process recordings following auto-decryption? (traditional): Select the slower 'traditional' automatic processing following decryption
Folder flag, Sweeper or No automatic processing of recordings: Select this option if you are using 'Auto Ad detection' folder flags, or Sweeper to control DetectAds or you want no automatic processing.
Exclude channels from automatic processing: Exclude BBC and other non commercial channels from processing, see below for more details
Start processing recording queue: Target time to start processing recording queue for Traditional Processing, ASAP or a time of day
Only Bookmark ends of ad breaks?: Select this option if you use the remote control to skip to the end of ad-breaks and do not need/want bookmarks at the start of ad-breaks. With this option you will not be able to subsequently use Crop manually to delete ad-breaks (without rerunning detection) but you can still use the automatic crop option.
Crop recording following ad detection?: Automatically run nicesplice to delete the adbreaks and shrink the recording file
Delete (move to bin) original recording after processing?: Select No if you don't trust the detection/cropping process 100% and want to keep the original around without hunting in the dustbin to find it.
Write output recording to folder: Choose target for writing output recording, 'Other' can only be a fixed path.
Limit cpu usage to: Option will only be shown if the cpulimit package has been installed and will limit the cpu usage of the ffmpeg program (cpu hog) to approximately the specified value. This will slow down traditional processing significantly if the percentage is less than 50%, the impact on chaserun is less significant unless you are simultaneously recording two HD channels.; This support is still experimental and it is not yet known if cpu limiting will reduce drop-outs and other faults - please report findings in the forums.
Other Options: You can specify any other Processing Options here that don't have specific fields (above) in the settings.

Channel exclusion list

You can type a generic matching rule such as 'BBC *' in the left hand or ctrl-click on specific channels and then click the ← button to transfer them to the left column. To remove entries from the left column position the cursor and use delete or backspace keys.

If you record from CBeebies, CBBC, etc I suggest you add them to the exclusion list since they are not covered by the default configuration.

ChaseGet settings

Options specified via the webif Settings section for ChaseGet are:

Hide ChaseGet on status display?: Hide/Show ChaseGet activity on the webif status display
Mute sound on ChaseGet wake from standby?: Muting the sound prevents unwanted disturbance if the TV is on (or auto switched on) when the system awakes from standby outside peak hours.

N.B. Do not use this option if your remote is programmed to only control TV sound and can't be used to send volume controls to the Humax

Don't mute or return to standby between and: Use this option to specify your peak viewing times to prevent the box returning to standby without warning while it is possible that family members may be watching.

To prevent ChaseGet ever returning the system to standby specify range as 00:00 to 24:00
To allow ChaseGet to return system to standby at any time specify the range as 00:00 to 00:00

Chase playing

You can watch the cropped output from the chaserun process whilst recording is still in progress either through the normal TV user interface or via file sharing using an application such as VLC however there are a number of things to be born in mind.

You can't remotely access the file via URL, only via file file sharing, there are two reasons for this.
- The output files have not been indexed by the DLNA server and
- the DLNA server only returns the amount of program known at the start of the request and so wouldn't return the entire program
If you use the skip/fast forward buttons you are liable to run past the end as known by the Humax and playback will terminate prematurely
If watching the -crop cropped output you have to take into account the total amount of advert breaks to be removed and the lag in producing cropped output. e.g. if you starting watching a program with 15 minutes of ads 10 minutes after it starts you will inevitably catch up with what has been processed before the end of program and playback will end. It would be safer to wait 30 minutes before attempting to start Chase play.
The full recording length will not be shown in the program information on the TV or via webif. This is because the Humax updates the resume time during playback overwriting the actual recording length in the process.
File will not be renamed to the original file name when recording finishes.

In general you will get the best Chase play experience viewing the -crop output on TV, starting half way or later through the recording time and leaving the remote control alone once play has started!

Limitations

DetectAds works by detecting the short periods of silence that occur between ads and looks for clusters of these silence to distinguish between ad breaks and natural silences occurring during programmes.

Unfortunately sponsor bumper messages, viewer competitions, news bulletins and some trailers don't always have the necessary silences to allow them to be detected and removed.

Some ad breaks, especially in overnight programs, are too short to be reliably detected.

Ads in radio transmissions are not detected, DetectAds automatic processing will exclude recordings that have the Radio flag set.

Debugging

DetectAds writes a few lines to the recmon.log depending on operating mode but the vast majority of output is written to /mod/tmp/auto.log for queued requests and to /mod/tmp/detectads.log for programmes processed during recording, records from chaseget are written to /mod/tmp/chaseget.log

Logs can be examined using the log viewer on the webif Diagnostics page or a text editor.

The amount of log data written is controlled by the "Auto-processing log level" setting in General Settings on the webif Settings page or specifying the -d option when using the command line interface.

If you experience a power cut or other failure during chaserun processing you may be left with orphaned -dec and -inp files in the /mod/tmp/ directory. These should not cause problems and will automatically be deleted.

Under the covers

In chaserun mode DetectAds sets up a pipeline where the stdout output from one program is fed into the stdin input of the next allowing them to run is parallel reducing overall elapsed time.

The stages in the chaserun pipeline are:

chaseget: This is a newly written jim program to retrieve and decrypt the input recording. It uses the system dlnahelper function to fool the DLNA server into allowing unindexed recordings to be retrieved. Curl is used to retrieve the recording in chunks with the pause between chunks allowing multiple instances of DetectAds when there are simultaneous recordings. Since the DLNA server is not active in half-awake recording state chaseget uses the IR package to force the system fully awake if needed.
tee: A standard linux utility is used to write the decrypted recording to the -dec file which is the input to the nsplice stage if cropping or the DetectAds output if not cropping.
ffmpeg: A general purpose audio and video stream processor. Extracts the audio component from the recording.
silence: Originally created for mythtv it detects commercial breaks by looking for periods of silence and then finding clusters of silences, the algorithm is described on the Myth TV Commercial_detection_with_silences page. Only minor changes have been made to sharpen the detection of silences and provide periodic heartbeat messages to help with cropping in the pipeline.
detectads: Part of the DetectAds jim code analyzes the messages written by silence, updates the bookmarks in the hmt the input & -dec files, and builds cut commands for nicesplice
nsplice: (optional)This is a version of the nicesplice program modified to accept commands from stdin instead of the command line and to make the output recording visible as it is processed to permit chase playing. The .nts for the -dec file input to splice is a hard link to the .nts for the input recording.

The traditional version of DetectAds has a similar flow except that chaseget & tee are omitted, ffmpeg reads the already decrypted input file directly, The other major difference is that nsplice runs after the pipeline has completed rather than being a pipeline stage.

Change History

Date	Version	Author	Description
2019-01-18	detectads 0.2.5-4	mymsman	Minor update Fix Run Now webif option
2019-01-06	detectads 0.2.5-3 chaseget 0.1.2-1	mymsman	Minor update System notify (red warning in weibif)if Content Sharing not enabled DetectAds Defers entry on Auto process queue ChaseGet wait max 10 minutes for DLNA to become active
2018-12-22	chaseget 0.1.2-0	mymsman	Minor update Don't crash on settings failure Logging improvements Updates to conf file for new auto process Run standby scan at 13, 33, 53 minutes past hour
2018-09-30	detectads 0.2.5-0	mymsman	Significant update Expose all options Thumbnail creation Reset flag and bookmarks option Reworking of wiki pages
2017-01-30	detectads 0.2.4-1	mymsman	Significant update It now uses the system queue for all deferred requests rather than maintaining its own database The old queue will be processed if there are outstanding requests on it but no new entries will be added and it will eventually be deleted. You can add Ad Detection requests to the queue using the Browse and Sweeper 'Queue for' options The old DetectAds Sweeper actions remain for compatibility with existing rules but now use the system queue The sample Sweeper rule has been updated to use the 'Queue recording for' action There are two new folder flags Auto DetectAds and No DetectAds These work in conjunction with the existing Channel exclusions rules so it is not necessary to add the flag to each directory if you use the existing DetctAds Auto processing options (Automatically process whilst recording in progress? (chaserun) or Automatically process recording following auto-decryption? (traditional) ) You can use these flags to include additional directories (Auto DetectAds) where you want all (including pre-existing recordings) ad detected or exclude directories (No DetectAds) which would otherwise be subject to ad detection based on channel filters. For instance you could set No DetectAds on the 'My Videos' root folder to prevent auto ad detection of one-off recordings (I don't recommend setting Auto DetectAds on 'My Videos' since this would override the channel selection filters and cause detection of all one-off recordings including those on BBC channels) DetectAds processing can still be deferred to, for example overnight, by setting a target start time on the settings page (remember to leave machine on or set a reminder) Logging for Queued requests will now be in auto.log but remain in detectads.log for detect while recording. Chaseget logging will be in chaseget.log. Chaserun processing remains a process while recording operation though it does create a temporary queue entry as a safeguard against failures during recording.
2016-11-16	detectads 0.2.3-3 chaseget 0.1.1-6	mymsman	Minor update Show DetectAds and ChaseGet activity in webif status display Option on settings menu to hide ChaseGet status display
2016-05-25	detectads 0.2.3-2 chaseget 0.1.1-5	mymsman	Medium update New option to specify target start time for queued requests: On Settings page On Browse detectads page As sweeper parameter -t hh:mm via command line -t hh:mm or -t timevalue Minor cleanup of settings page Sweeper rules restored to Global Chaseget now uses system uptime to determine boot time
2016-03-28	detectads 0.2.3-0 chaseget 0.1.1-2	mymsman	Medium update If cpulimit package is installed: New settings option to specify limit Limits ffmpeg usage to specified value Automatic clean up of orphaned files Detect and log 'pipe broken' in chaseget 'Back to media browser' should work in flexview Sweeper rules no longer Global
2016-03-20	detectads 0.2.2-7	mymsman	Minor update Bypass detection for Radio programmes with Radio flag set in hmt Remove Radio filter from default configuration. Include flexview.hook file
2015-11-30	detectads 0.2.2-6	mymsman	Minor update Automatic restart of detectads following power failures during Detect while recording Decryption length failures re-queued after recording ends Reduced max ad length to 65 seconds (was 90 sec) Heartbeat at cluster start - should improve chase playing
2015-11-23	detectads 0.2.2-5	mymsman	Minor update Avoid too frequent re queuing on repeated decryption length errors
2015-11-15	detectads 0.2.2-3	mymsman	Minor update Fixes the bug that caused misplaced crops when using chaserun on HD files Detects when poor quality recording cause a significant (>5 seconds) difference between detected program length and actual program length Create System notify messages when errors are detected
2015-10-28	detectads 0.2.2-1	mymsman	Experimental update Use links to input file to 'hide' from DLNA indexing
2015-10-20	detectads 0.2.2-0 chaseget 0.1.1-1	mymsman	Significant update ChaseGet return to standby ChaseGet settings page 'Inverted' bookmarks Retain original file name/title on output file Automatic reprocess on significant length error Build -dec file in /mod/tmp/ with hard link for .nts
2015-09-30	detectads 0.2.1-6	mymsman	Minor update Recompile Silence to work with updated libsndfile package Use -ac 2 option on ffmpeg These changes should provide better large file support
2015-09-26	detectads 0.2.1-4	mymsman	Bug fix Fix incorrect dependency
2015-09-25	detectads 0.2.1-3	mymsman	Bug fix Fix for short crop problem
2015-09-23	detectads 0.2.1-2	mymsman	Bug fix Fix exclusion of BBC, more diagnostics for short crop problem
2015-09-17	detectads 0.2.1-1	mymsman	Minor update Show Addetection icon in browse file list (thanks af123)
2015-09-16	detectads 0.2.1-0	mymsman	Significant update Addetection flag in .hmt file (thanks af123) Automation via Sweeper (thanks af123) Sample Sweeper rules Show processing queue, remove entries from queue Only Bookmark ends of ad breaks option New icon Cosmetic improvements No deletion of inuse recordings
2015-09-01	detectads 0.2.0-3	mymsman	Minor update Add ffmpeg and libsndfile to prereq list Correct bug if no ad-breaks found
2015-08-31	detectads 0.2.0-2 chaseget 0.1.0-0	mymsman	Released on package catalogue ChaseGet moved to separate package .conf & .db moved to /mod/etc/ Avoid power on loop Minor clean up
2015-08-01	detectads 0.2.0-0	mymsman	Significant update including: Channel exclusion Background processing from browse panel Command line Automatic cropping of breaks option Detect whilst recording Limited distribution
2014-07-04	detectads 0.0.2.2	njm	Initial version including while you wait webif and process all automation option

Future Enhancements

Possible future enhancements - please suggest more

Replace use of ffmpeg for audio extraction
Attempt to reduce sound and audio dropouts at cut points

Acknowledgements

njm: For having the idea of bringing the mythTV ad detection to the Humax and for creating the first version of DetectAds
mythTV: For producing the Silence ad detecting package Myth TV Commercial_detection_with_silences
oatcake: For coming up with the idea of using symbolic links to allow DLNA access to files that have not yet been DLNA indexed
af123: For providing the recmon start recording hooks and the webif framework, adding the Addetection flag and Sweeper hooks.
xyz321: Automatic inversion of bookmarks, libsndfile improvements
drutt: For creating nicesplice package and permitting me to update the source.
everbody else: Who has assisted through the forums

Difference between revisions of "DetectAds"