Difference between revisions of "Sweeper"

From hummy.tv Wiki
Jump to: navigation, search
(Created page with "== Sweeper - the Swiss Army Knife of recording management == === Available actions === ; continue : do nothing and continue to next rule. ; stop : do nothing and stop process...")
 
m (Tautology removal!)
 
(73 intermediate revisions by one other user not shown)
Line 1: Line 1:
 
== Sweeper - the Swiss Army Knife of recording management ==
 
== Sweeper - the Swiss Army Knife of recording management ==
 +
 +
{| border="0"
 +
| [[File:Sweeper256.png]]
 +
| __TOC__
 +
|}
 +
 +
=== Overview ===
 +
 +
Sweeper is a Custom Firmware package for managing recordings in a variety of ways using custom rules. It started life as a simple utility for tidying up one-off recordings in the top level of the ''My Video'' area into folders, hence its name, but has since evolved into a multi-purpose tool.
 +
 +
Once installed, the rules that apply to the recordings and series folders at the top level of the ''My Video'' area can be accessed via the ''Sweeper'' icon on the main menu or toolbar. It is also possible to define rules for other folders by selecting ''Sweeper Rules'' from the OPT+ menu alongside the folder in the web interface.
 +
 +
=== User Interface ===
 +
 +
[[File:Sweeper_Main.png|800px]]
 +
 +
The main sweeper rule screen shows the rules which are currently defined for the folder followed by a set of buttons which can be used to add new rules, save or discard any changes, view or test the configuration and optionally show a textual representation of each rule, updated in real-time as changes are made.
 +
 +
At the bottom of the screen is an option to add a pre-defined ruleset to the rules already on screen. A set of rules can be chosen from the dropdown list and then added to the list of rules with a click on the ''Add to rules'' button; once there they can be individually modified, disabled or deleted as required.
 +
 +
All changes made are immediately reflected on screen but no changes are made to the system configuration until the ''Save changes'' button is pressed and confirmed.
 +
 +
=== Anatomy of a Rule ===
 +
[[File:Sweeper Rule.png|800px|Anatomy of a Sweeper Rule]]
 +
 +
Each rule is shown in a box with an icon to the top left indicating whether it is a file or folder rule...
 +
 +
=== Tokens ===
 +
 +
Several of the rule actions take parameters, and these parameters can contain tokens that are replaced with information from the recording which is being processed. For example, you could use the ''Move recording to folder'' action with an argument of ''%genre'' which would place Films in a folder called Films, Children's recordings in one called Children, etc.
 +
 +
The available tokens are shown below:
 +
 +
{| class="wikitable" style="border: 1px solid #ccc; cellspacing: 1, cellpadding: 5"
 +
! %orig
 +
| The original version of the item being modified. For example to append _backup to the filename in a ''setfilename'' action, you would set the filename to %orig_backup.
 +
|-
 +
! %title
 +
| Recording title.
 +
|-
 +
! %filename
 +
| Recording filename.
 +
|-
 +
! %basename
 +
| The base name of the recording filename ''(without the directory and file extension)''.
 +
|-
 +
! %folder
 +
| Complete folder name ''(including /media/My Video or equivalent)''.
 +
|-
 +
! %bfolder
 +
| Base Folder name ''(without /media/My Video or equivalent)''.
 +
|-
 +
! %synopsis
 +
| Recording synopsis.
 +
|-
 +
! %epname
 +
| The name of the episode as extracted from the recording synopsis. Reliant on the broadcasters providing the data.
 +
|-
 +
! %series
 +
| The number of the series (extracted from the synopsis, 0 if not found).
 +
|-
 +
! %episode
 +
| The episode number (extracted from the synopsis, 0 if not found).
 +
|-
 +
! %episodes
 +
| The number of episodes in the series (extracted from the synopsis, 0 if not found).
 +
|-
 +
! %epdescr
 +
| A representation of the series and episode data found in the synopsis, e.g. s15e5/10. Unknown fields are replaced with a ?
 +
|-
 +
! %channel
 +
| Channel name.
 +
|-
 +
! %lcn
 +
| Logical channel number.
 +
|-
 +
! %genre
 +
| Recording genre.
 +
|-
 +
! %definition
 +
| Definition of recording (HD or SD).
 +
|-
 +
! %duration
 +
| Recording duration (in minutes).
 +
|-
 +
! %timestamp
 +
| Recording start time as YYYYMMDDHHMMSS.
 +
|-
 +
! %yyyymmmdd
 +
| Recording start time as YYYYMMDD.
 +
|-
 +
! %hhmm
 +
| Recording start time as HHMM.
 +
|-
 +
! %hh
 +
| Recording start hour as HH.
 +
|-
 +
! %mm
 +
| Recording start minute as MM.
 +
|-
 +
! %year
 +
| Recording start year as YYYY.
 +
|-
 +
! %2digityear
 +
| Recording start year as YY.
 +
|-
 +
! %month
 +
| Recording start month number 1-12.
 +
|-
 +
! %2digitmonth
 +
| Recording start month number 01-12.
 +
|-
 +
! %date
 +
| Recording start date 1-31.
 +
|-
 +
! %2digitdate
 +
| Recording start date 01-31.
 +
|-
 +
! %shortday
 +
| Recording start day (short form, e.g. Mon)
 +
|-
 +
! %longday
 +
| Recording start day (long form, e.g. Monday)
 +
|-
 +
! %shortmonth
 +
| Recording start month (short form, e.g. Jan)
 +
|-
 +
! %longmonth
 +
| Recording start month (long form, e.g. January)
 +
|-
 +
! %etimestamp
 +
| Recording end time as YYYYMMDDHHMMSS.
 +
|-
 +
! %eyyyymmmdd
 +
| Recording end time as YYYYMMDD.
 +
|-
 +
! %ehhmm
 +
| Recording end time as HHMM.
 +
|-
 +
! %ehh
 +
| Recording end hour as HH.
 +
|-
 +
! %emm
 +
| Recording end minute as MM.
 +
|-
 +
! %replace:''string'':''search'':''replace'':
 +
| Perform a substitution on ''string'' replacing ''search'' with ''replace''. The delimiter (shown here as :) can be anything as long as the same character is used in all three places. Requires Sweeper version 2.1.1.
 +
|-
 +
! %regsub:''string'':''search'':''replace'':
 +
| As ''replace'' but supports regular expressions. Requires Sweeper version 2.1.1.
 +
|-
 +
! %asfilename:''string'':
 +
| Convert the argument to a filename as the Humax software would, replacing reserved characters with underscores.
 +
|-
 +
! %asuniqfilename:''string'':
 +
| As above but if the resulting filename already exists then append a number to make it unique. Requires Sweeper version 2.1.1.
 +
|-
 +
! %format:''formatspec'':''string'':
 +
| Format a string using a Jim format specification.<br>Example: ''%format:%06d:%episode:'' returns the episode number padded with leading zeros to make it up to 6 characters.<br>Example: ''%format:08b:%episode:'' returns the episode number in binary.
 +
|-
 +
! %%''variablename''
 +
| Replace with the contents of the named variable (set previously with a '''set''' action).
 +
|-
 +
! %replace:''search'':''replace'':
 +
| <font color=red>Deprecated, use three argument version instead.</font> Perform a substitution on the result replacing ''search'' with ''replace''. The delimiter (shown here as :) can be anything as long as the same character is used in all three places.
 +
|-
 +
! %regsub:''search'':''replace'':
 +
| <font color=red>Deprecated, use three argument version instead.</font> As ''replace'' but supports regular expressions.
 +
|}
  
 
=== Available actions ===
 
=== Available actions ===
; continue : do nothing and continue to next rule.
+
 
; stop : do nothing and stop processing rules.
+
{| class="wikitable" style="border: 1px solid #ccc; cellspacing: 1, cellpadding: 5"
; preserve : (deprecated) do nothing and stop processing rules.
+
|-
; move : move the recording to a folder.
+
! Action
; movecreate : move the recording to a folder and create it if it doesn't exist.
+
! Description
; fileunder (folders only) : look for another existing with the same name as the new series recording and, if found, move the new recordings into it.
+
! Parameter
; fileundercreate (folders only) : As fileunder but if no folder is found then create a new one.
+
|-
 +
! continue
 +
| Do nothing and continue to next rule.
 +
| ''None''
 +
|-
 +
! stop
 +
| Do nothing and stop processing rules for this folder.
 +
| ''None''
 +
|-
 +
! preserve
 +
| (deprecated - use ''stop'' instead) do nothing and stop processing rules.
 +
| ''None''
 +
|-
 +
! lock
 +
| Lock recording and continue to next rule.
 +
| ''None''
 +
|-
 +
! unlock
 +
| Unlock recording and continue to next rule.
 +
| ''None''
 +
|-
 +
! move
 +
| Move the recording to a folder.
 +
| Folder name relative to the system media root, e.g. (Films/Children)
 +
|-
 +
! movecreate
 +
| move the recording to a folder and create it if it doesn't exist.
 +
| Folder name relative to the system media root, e.g. (Films/Children)
 +
|-
 +
! renamefile
 +
| rename the files that make up the recording and continue to the next rule. The argument will usually contain tokens that are replaced with values from the recording being processed.
 +
| New filename base (no extension) e.g. %title_%yyyymmdd_%hhmm
 +
|-
 +
! settitle
 +
| set the media list title for the recording. Tokens can be used and the %orig token contains the current title.
 +
| New title
 +
|-
 +
! setguidance
 +
| set the guidance text for the recording.
 +
| New guidance text or blank to remove any guidance rating for the recording.
 +
|-
 +
! delete
 +
| delete the recording or, if ''undelete'' is installed, move it to the bin.
 +
| ''None''
 +
|-
 +
! fileunder
 +
| Folders only; look for another existing with the same name as the new series recording and, if found, move the new recordings into it.
 +
| Folder name from which to start the search (e.g. Films) - leave blank to search from the top level.
 +
|-
 +
! fileundercreate
 +
| Folders only; as fileunder but if no folder is found then create a new one.
 +
| Folder name from which to start the search (e.g. Films) - leave blank to search from the top level.
 +
|-
 +
! flag
 +
| Set a flag in the recording file.
 +
| A flag name such as 'dedup', 'shrunk', 'encrypted'
 +
|-
 +
! unflag
 +
| Unset a flag in the recording file.
 +
| A flag name such as 'dedup', 'shrunk', 'encrypted'
 +
|-
 +
! log
 +
| Output a log message (predominantly used for testing).
 +
| The message to be logged.
 +
|-
 +
! queue
 +
| Add the recording file to the queue named in the argument.
 +
|-
 +
! set
 +
| Set a variable to a value.
 +
| variablename=value
 +
|}
  
 
=== Available conditions ===
 
=== Available conditions ===
; flag : a flag against the recording, e.g. 'flag New'. Flags are Locked, New, Encrypted, Guidance, ODEncrypted, Shrunk, Deduped.
 
; lcn : Logical channel number, e.g. 'lcn 1', 'lcn {> 1}'
 
; duration : recording duration in minutes, e.g. 'duration {> 60}'
 
; schedduration : scheduled recording duration in minutes.
 
; size : size in bytes
 
; age : age in hours (measured from recording end time).
 
; wage : number of hours since last watched, or from recording end time if not watched.
 
; definition : standard (SD) or high (HD) definition, e.g. 'definition SD'
 
; title : title contains ..., e.g. 'title {The Big Bang Theory}'
 
; synopsis : synopsis contains ...
 
; guidance : guidance contains ...
 
; genre : recording genre is.. e.g. 'genre Children'
 
; hour : hour of recording, e.g. 'hour {>= 21}'
 
  
=== Available tokens ===
+
{| class="wikitable" style="border: 1px solid #ccc; cellspacing: 1, cellpadding: 5"
 +
! lastrule
 +
| True if the last rule was successfully matched.
 +
|-
 +
! flag
 +
| A flag against the recording, e.g. 'flag New'. Flags are Locked, New, Encrypted, Guidance, ODEncrypted, Shrunk, Deduped.
 +
|-
 +
! lcn
 +
| Logical channel number, e.g. 'lcn 1', 'lcn {> 1}'
 +
|-
 +
! duration
 +
| Recording duration in minutes, e.g. 'duration {> 60}'
 +
|-
 +
! schedduration
 +
| Scheduled recording duration in minutes.
 +
|-
 +
! size
 +
| Size in bytes
 +
|-
 +
! age
 +
| Age in hours (measured from recording end time).
 +
|-
 +
! wage
 +
| Number of hours since last watched, or from recording end time if not watched.
 +
|-
 +
! definition
 +
| Standard (SD) or high (HD) definition, e.g. 'definition SD'
 +
|-
 +
! title
 +
| Title contains ..., e.g. 'title {The Big Bang Theory}'
 +
|-
 +
! synopsis
 +
| Synopsis contains ...
 +
|-
 +
! guidance
 +
| Guidance contains ...
 +
|-
 +
! genre
 +
| Recording genre is.. e.g. 'genre Children'
 +
|-
 +
! hour
 +
| Hour of recording, e.g. 'hour {>= 21}'
 +
|-
 +
! now
 +
| Current time in 24-hour format, e.g. 2359
 +
|-
 +
! filename
 +
| Filename on disk contains ...
 +
|-
 +
! fflag
 +
| Folder containing the recording has a particular flag, e.g. 'fflag autodecrypt'
 +
|-
 +
! foldername
 +
| Folder name contains...
 +
|-
 +
! series
 +
| Folder rules only - true if folder was automatically created due to a scheduled recording.
 +
|-
 +
! bookmarks
 +
| Number of bookmarks
 +
|-
 +
! fileexists
 +
| Does the file specified exist? If the argument is not a complete pathname then it is taken to be relative to the recording's directory.
 +
|-
 +
! direxists
 +
| Does the directory specified exist?  If the argument is not a complete pathname then it is taken to be relative to the recording's directory.
 +
|-
 +
! queue
 +
| Test whether the current recording is queued for the operation named in the parameter.
 +
|-
 +
! textmatch
 +
| Match an arbitrary string against a pattern. The string and pattern are separated with ''~~'', e.g. 'textmatch {%synopsis~~*drama*}'
 +
|-
 +
! intmatch
 +
| Match an arbitrary string against a numeric test. The string and test are separated with ''~~'', e.g. 'intmatch {%year~~> 2014}'. There must be a space between the operator and the test value.
 +
|-
 +
! varset
 +
| Test whether the named variable has been set.
 +
|-
 +
! or
 +
| True if any of the arguments are true, e.g. 'or {lcn 1 lcn 2 lcn 3}'
 +
|-
 +
! and
 +
| True if all arguments are true, e.g. 'and {lcn >= 1 lcn <= 3}'
 +
|}
 +
 
 +
In the configuration file, all conditions can be optionally prefixed with a ! to indicate negation. For example, the condition ''!lcn 1'' means '''not''' recorded from channel 1.
 +
 
 +
=== String matching ===
 +
 
 +
When a condition takes a string argument for a match, that can either be a fixed string, a pattern or a regular expression. A fixed string is treated as a substring match - it is just looked for anywhere in the attribute. A regular expression is specified by prefixing it with a ~ character. A pattern is matched against the whole attribute and can contain the following special sequences:
  
; %genre : Recording genre.
+
; *
; %definition : Definition of recording (HD or SD).
+
: Matches any sequence of characters in string, including a null string.
; %title : Recording title.
+
; ?
; %channel : Channel name.
+
: Matches any single character in string.
; %lcn : Logical channel number.
+
; [chars]
; %duration : Recording duration (in minutes).
+
: Matches any character in the set given by chars. If a sequence of the form x-y appears in chars, then any character between x and y, inclusive, will match.
; %timestamp : Start of recording as YYYYMMDDHHMMSS.
+
; \x
; %yyyymmmdd: Start of recording as YYYYMMDD.
+
: Matches the single character x. This provides a way of avoiding the special interpretation of the characters \*?[] in pattern.
; %hhmm: Start of recording as HHMM
 

Latest revision as of 14:24, 27 October 2020

Sweeper - the Swiss Army Knife of recording management

Sweeper256.png

Overview

Sweeper is a Custom Firmware package for managing recordings in a variety of ways using custom rules. It started life as a simple utility for tidying up one-off recordings in the top level of the My Video area into folders, hence its name, but has since evolved into a multi-purpose tool.

Once installed, the rules that apply to the recordings and series folders at the top level of the My Video area can be accessed via the Sweeper icon on the main menu or toolbar. It is also possible to define rules for other folders by selecting Sweeper Rules from the OPT+ menu alongside the folder in the web interface.

User Interface

Sweeper Main.png

The main sweeper rule screen shows the rules which are currently defined for the folder followed by a set of buttons which can be used to add new rules, save or discard any changes, view or test the configuration and optionally show a textual representation of each rule, updated in real-time as changes are made.

At the bottom of the screen is an option to add a pre-defined ruleset to the rules already on screen. A set of rules can be chosen from the dropdown list and then added to the list of rules with a click on the Add to rules button; once there they can be individually modified, disabled or deleted as required.

All changes made are immediately reflected on screen but no changes are made to the system configuration until the Save changes button is pressed and confirmed.

Anatomy of a Rule

Anatomy of a Sweeper Rule

Each rule is shown in a box with an icon to the top left indicating whether it is a file or folder rule...

Tokens

Several of the rule actions take parameters, and these parameters can contain tokens that are replaced with information from the recording which is being processed. For example, you could use the Move recording to folder action with an argument of %genre which would place Films in a folder called Films, Children's recordings in one called Children, etc.

The available tokens are shown below:

%orig The original version of the item being modified. For example to append _backup to the filename in a setfilename action, you would set the filename to %orig_backup.
%title Recording title.
%filename Recording filename.
%basename The base name of the recording filename (without the directory and file extension).
%folder Complete folder name (including /media/My Video or equivalent).
%bfolder Base Folder name (without /media/My Video or equivalent).
%synopsis Recording synopsis.
%epname The name of the episode as extracted from the recording synopsis. Reliant on the broadcasters providing the data.
%series The number of the series (extracted from the synopsis, 0 if not found).
%episode The episode number (extracted from the synopsis, 0 if not found).
%episodes The number of episodes in the series (extracted from the synopsis, 0 if not found).
%epdescr A representation of the series and episode data found in the synopsis, e.g. s15e5/10. Unknown fields are replaced with a ?
%channel Channel name.
%lcn Logical channel number.
%genre Recording genre.
%definition Definition of recording (HD or SD).
%duration Recording duration (in minutes).
%timestamp Recording start time as YYYYMMDDHHMMSS.
%yyyymmmdd Recording start time as YYYYMMDD.
%hhmm Recording start time as HHMM.
%hh Recording start hour as HH.
%mm Recording start minute as MM.
%year Recording start year as YYYY.
%2digityear Recording start year as YY.
%month Recording start month number 1-12.
%2digitmonth Recording start month number 01-12.
%date Recording start date 1-31.
%2digitdate Recording start date 01-31.
%shortday Recording start day (short form, e.g. Mon)
%longday Recording start day (long form, e.g. Monday)
%shortmonth Recording start month (short form, e.g. Jan)
%longmonth Recording start month (long form, e.g. January)
%etimestamp Recording end time as YYYYMMDDHHMMSS.
%eyyyymmmdd Recording end time as YYYYMMDD.
%ehhmm Recording end time as HHMM.
%ehh Recording end hour as HH.
%emm Recording end minute as MM.
%replace:string:search:replace: Perform a substitution on string replacing search with replace. The delimiter (shown here as :) can be anything as long as the same character is used in all three places. Requires Sweeper version 2.1.1.
%regsub:string:search:replace: As replace but supports regular expressions. Requires Sweeper version 2.1.1.
%asfilename:string: Convert the argument to a filename as the Humax software would, replacing reserved characters with underscores.
%asuniqfilename:string: As above but if the resulting filename already exists then append a number to make it unique. Requires Sweeper version 2.1.1.
%format:formatspec:string: Format a string using a Jim format specification.
Example: %format:%06d:%episode: returns the episode number padded with leading zeros to make it up to 6 characters.
Example: %format:08b:%episode: returns the episode number in binary.
%%variablename Replace with the contents of the named variable (set previously with a set action).
%replace:search:replace: Deprecated, use three argument version instead. Perform a substitution on the result replacing search with replace. The delimiter (shown here as :) can be anything as long as the same character is used in all three places.
%regsub:search:replace: Deprecated, use three argument version instead. As replace but supports regular expressions.

Available actions

Action Description Parameter
continue Do nothing and continue to next rule. None
stop Do nothing and stop processing rules for this folder. None
preserve (deprecated - use stop instead) do nothing and stop processing rules. None
lock Lock recording and continue to next rule. None
unlock Unlock recording and continue to next rule. None
move Move the recording to a folder. Folder name relative to the system media root, e.g. (Films/Children)
movecreate move the recording to a folder and create it if it doesn't exist. Folder name relative to the system media root, e.g. (Films/Children)
renamefile rename the files that make up the recording and continue to the next rule. The argument will usually contain tokens that are replaced with values from the recording being processed. New filename base (no extension) e.g. %title_%yyyymmdd_%hhmm
settitle set the media list title for the recording. Tokens can be used and the %orig token contains the current title. New title
setguidance set the guidance text for the recording. New guidance text or blank to remove any guidance rating for the recording.
delete delete the recording or, if undelete is installed, move it to the bin. None
fileunder Folders only; look for another existing with the same name as the new series recording and, if found, move the new recordings into it. Folder name from which to start the search (e.g. Films) - leave blank to search from the top level.
fileundercreate Folders only; as fileunder but if no folder is found then create a new one. Folder name from which to start the search (e.g. Films) - leave blank to search from the top level.
flag Set a flag in the recording file. A flag name such as 'dedup', 'shrunk', 'encrypted'
unflag Unset a flag in the recording file. A flag name such as 'dedup', 'shrunk', 'encrypted'
log Output a log message (predominantly used for testing). The message to be logged.
queue Add the recording file to the queue named in the argument.
set Set a variable to a value. variablename=value

Available conditions

lastrule True if the last rule was successfully matched.
flag A flag against the recording, e.g. 'flag New'. Flags are Locked, New, Encrypted, Guidance, ODEncrypted, Shrunk, Deduped.
lcn Logical channel number, e.g. 'lcn 1', 'lcn {> 1}'
duration Recording duration in minutes, e.g. 'duration {> 60}'
schedduration Scheduled recording duration in minutes.
size Size in bytes
age Age in hours (measured from recording end time).
wage Number of hours since last watched, or from recording end time if not watched.
definition Standard (SD) or high (HD) definition, e.g. 'definition SD'
title Title contains ..., e.g. 'title {The Big Bang Theory}'
synopsis Synopsis contains ...
guidance Guidance contains ...
genre Recording genre is.. e.g. 'genre Children'
hour Hour of recording, e.g. 'hour {>= 21}'
now Current time in 24-hour format, e.g. 2359
filename Filename on disk contains ...
fflag Folder containing the recording has a particular flag, e.g. 'fflag autodecrypt'
foldername Folder name contains...
series Folder rules only - true if folder was automatically created due to a scheduled recording.
bookmarks Number of bookmarks
fileexists Does the file specified exist? If the argument is not a complete pathname then it is taken to be relative to the recording's directory.
direxists Does the directory specified exist? If the argument is not a complete pathname then it is taken to be relative to the recording's directory.
queue Test whether the current recording is queued for the operation named in the parameter.
textmatch Match an arbitrary string against a pattern. The string and pattern are separated with ~~, e.g. 'textmatch {%synopsis~~*drama*}'
intmatch Match an arbitrary string against a numeric test. The string and test are separated with ~~, e.g. 'intmatch {%year~~> 2014}'. There must be a space between the operator and the test value.
varset Test whether the named variable has been set.
or True if any of the arguments are true, e.g. 'or {lcn 1 lcn 2 lcn 3}'
and True if all arguments are true, e.g. 'and {lcn >= 1 lcn <= 3}'

In the configuration file, all conditions can be optionally prefixed with a ! to indicate negation. For example, the condition !lcn 1 means not recorded from channel 1.

String matching

When a condition takes a string argument for a match, that can either be a fixed string, a pattern or a regular expression. A fixed string is treated as a substring match - it is just looked for anywhere in the attribute. A regular expression is specified by prefixing it with a ~ character. A pattern is matched against the whole attribute and can contain the following special sequences:

*
Matches any sequence of characters in string, including a null string.
 ?
Matches any single character in string.
[chars]
Matches any character in the set given by chars. If a sequence of the form x-y appears in chars, then any character between x and y, inclusive, will match.
\x
Matches the single character x. This provides a way of avoiding the special interpretation of the characters \*?[] in pattern.