CREATING FILTERS
Multi-threaded NewsWatcher

Contents

 Go back to the page on filtering.
Go to the page on regular expressions.

CREATING FILTERS
Creating filters You can create a new filter in a number of ways, some of which specify initial settings for the new filter.
When a list of groups is showing on screen, hilite a group that you want to make a filter for. Go to the 'Filter' submenu on the Edit menu, and you will see 'Add Group Filter...', and another submenu with a list of the group hierarchies to which the current group belongs. Choose any one of these, or the original 'Add Group Filter...' item, and a dialog box appears on your screen:
The Group Filter Edit dialog box, showing the one filter defined for comp.sys.mac.comm, and seven global filters (defined for *). You can choose to view filters defined at a different level (e.g. comp.sys.*) using the popup menu above the lower panel.

In the lower left are a couple of radio buttons that specify whether articles in this group that are not matched by any filters are kept, or deleted.

In this case, I chose to add a filter for the group comp.sys.mac.comm. Filters which have already been created for this group are shown in the top panel. In the lower panel are shown more general filters, which apply to one of the hierarchies to which comp.sys.mac.comm belongs, for example comp.sys.mac.* (all the Mac groups) or comp.* (all the computing groups). The group level '*' refers to all groups, so filters created in this category will be global filters. You can move between levels in the parent hierarchy using the popup menu above the lower panel. We want to create a new filter for comp.sys.mac.comm, so click the 'New' button to the right of the upper panel. This brings up the Filter Edit dialog, which is shown below.
The dialog box used to edit individual filters. In this case, I'm creating a filter which will use a regular expression to hilite articles which are posted by people with 'apple.com'.
The filter dialog This dialog box contains two panels, between which you can navigate using the popup menu at the top of the dialog, or Command-up arrow and -down arrow keys. The first panel (above) specifies the specifies the matching criterion for this filter and its action, i.e. what happens to articles that are matched by this filter.

The second panel (below) allows you to specify whether the filter will expire, to control the section of the group hierarchy to which the filter applies (i.e. it scope), and various other options.

These various options are described in more detail in the following sections.

The second panel of the filter edit dialog, showing the filter's scope and various other options.

Filter actions

 The main actions of this filter are specified in the top section of the dialog box. These actions affect the status of the article (killed, hilighted etc), whether it is hilighted with color, and the assignment of a priority to the ariticle.

The popup in the top left specifies the main action of the filter:

  • Hilite
    Matching articles are colored in the subject list (as in the first image above), using the colors specified in the two color swatches in this dialog.

  • Keep
    Matching articles are not colored, but are simply kept in the list, useful if you are set a group to delete unmatched filters (discussed above).

  • Kill
    Matching articles are killed, in other words marked as read by the software before you ever see them. This is useful to remove those really annoying 'Make money fast' posts, and other threads that you get fed up of seeing.
Assigning priorities Articles matched by a filter are assigned some priority, or score, based on the settings of the applicable filters. Scores are used for sorting and possibly killing articles. The defined priorities are as follows:
  • Highest
  • Very High
  • High
  • Intermediate
  • Normal
  • Low
  • Very Low
N.B. These default priorities and their scores will be editable in future versions of MT-NewsWatcher.

By default, articles which are not matched by any filters are assigned a priority of 'Normal' and a score of 0; matched articles thence have their priorities raised or lowered by the applied filters. Filters with low scores will cause articles to move lower in the list, and those with higher scores to move towards the top. These effects are cumulative over the filters that apply to an article.

Coloring articles 'Hilite' filters color the subject or author text of matching articles in the main subject window, to draw attention to such articles.

Killed articles usually are not displayed, and so coloring them does not appear to make much sense. But you may make use of the 'Show killed articles' option in the Preferences, and perhaps assign light colors so that such articles are visible, but not prominent in the list.

You can choose fore- and background colors by clicking in the patches of color in the dialog; a palette of colors will pop up, from which you can choose a color. The palette also contains some empty swatches at the bottom, and an 'Other...' option that allows you to choose a color from the color picker. Colors chosen in this way then appear in the empty swatches, so that you can easily get at them again.

You can also change the colors in the body of the palette by holding down the option key when choosing a color. This again shows the color picker, and the color you choose then replaces the color in the palette at that location.

Matching criteria

 You specify the matching criterion for a filter using the controls in the first panel of the dialog box. You can search (using several different methods) for text in textual news message headers, or match articles by their line count, or posting date.
Filtering
text-based headers 		The first popup menu determines the action of this filter on matched articles: Hilite, Keep and Kill. The second popup menu specifies which header to search in for the text; the author, subject, organization etc. You will probably use filters based Subject most often, but it can also be useful to search other fields. Beware that choice of field here can lead to significantly longer downloading times.

Different types of text match are available in the next popup down, that shows 'Contains the string' in the picture. These are:

  • Contains
  • Does not contain
  • Starts with
  • Ends with
  • Is
  • Is not
  • Contains the word,
  • Contains a word beginning
  • Contains a word ending
  • Matches the Regular Expression
  • Does not match the Regular Expression
These matching types should be self-explanatory, and work in a similar way to the 'Find File' program. The last two matching types are based on Regular Expressions, which are a concise (and rather obtuse) means of matching strings using certain syntatical rules. See the page on Regular Expressions for more detail.
Filtering by lines MT-NewsWatcher allows you some quite sophisticated filtering based on the length, and date of articles. If you choose to filter by line counts, you see this panel in the Filter Edit dialog box:

and can specify to match articles longer, shorter, exactly, or not exactly the given number of lines. The 'Less than' and 'More than' comparisons are exclusive; 'Less than 2 lines' means that articles with 1 or fewer lines will be matched.
Filtering by dates Filtering by date provides two options; relative and absolute date matching. Relative date matching filters articles according to some age relative to the present, for example you can match articles older, or younger than 3 days:

When you use the 'Exactly' and 'Not exactly' comparisons to match dates, the outcome is dependent on the date measure (days, weeks etc) that you use. If you create a filter to hilite articles that are exactly 2 days old, MT-NewsWatcher subtracts two days from the current date, and selects articles that were posted on that day. If you filter by a number of months, then articles posted during the relevant month will be selected.

You can alternatively specify an absolute date, and show articles posted before or after this date:

In this case the match is done to a resolution of one day; all articles posted on the specified day will be selected.

Scoring

 Using filters to hilite articles with colors, or to kill unwanted articles, gives you control over what set of articles you see, and which of those are more in your face. Finer control over the prioritization of articles is possible through the use of scores, described next.

A score is an arbitrary value that a filter assigns to each article that it matches. As articles are filtered, they accumulate their own score, which is simply the sum of the scores of applied filters. Article scores can then be used to sort articles in subject windows, or for killing articles.

Scoring articles A set of predefined scores is available through the 'Filter priority' popup menu, in the Edit Filter dialog box (see picture above), ranging from Lowest (with a score of -10) to Highest, with a score of 100. Choosing a filter priority from this popup menu automatically assigns the corresponding score. The filter priority has no other function than to assign a score.

You can edit the score manually, simply by typing a value in the box beside the priority popup menu. If the score you typed in matches one of the predefined priorities, then the popup menu is updated to reflect this.

Filter scores can range from -32767 to 32767. Negative scores cause articles to be sorted lower in the list, and higher values sort them towards the top. Unfiltered articles have a score of 0 (zero), so you should use negative scores for those articles you want to assign an even lower priority.

You can show the final scores that articles end up with by checking the 'Show scores' checkbox in the 'Subject windows' preferences panel. This will display an extra column in subject windows with that score, as well as showing filter scores in the Filters window. This should provide enough information for you to verify that your scoring scheme is working as intended.

Killing by score For more control over killing articles, you can specify that those under a certains score are killed. To activate this function, check the relevant checkbox in the 'Filtering options' preferences panel, and supply the threshold score below which you wish to kill articles. For example, a threshold of 0 (zero) will kill all articles whose cumulative score is less than zero, while keeping unmatched articles with a score = 0.

Expiring filters

 Filters can be set to 'expire' after a certain length of time, which means that they become inactive, and are then deleted from your filters file. Expiring filters are useful for killing threads that you don't want to read any more, but which are not a permanent problem. Expiry date is controlled in the Edit Filter dialog, as shown here:

In this case, a filter was set up to expire 2 days after creation, the expiry date being shown below the controls. If you edit a filter again some time after creation, and want to extend its life by another period of time, you can choose that period by typing a number into the box, and choosing the 'From now' radio button. Then the expiry date is recalculated as the present, plus the time interval specified.

Those three different expiry options are as follows:

From creation date
The filter will expire the given time after it was created. This is the default option.
From now
Select this option to extend the life of the filter by another time period.
From last use
This option allows you to have 'self-expiring' filters, which will expire and disappear when they are no longer matching articles. With each filter is stored a date when it was last used (that is, when it last matched an article). On creation, this last used date is set to the creation date of the filter. So if a filter is no longer matching articles, once its 'last used' date is more than this length of time ago, the filter will be expired.

Filters with expiry dates are shown with a small 'watch' on the icon:

Although the time of expiry is saved to the nearest second, filters are only actually expired when you start up MT-NewsWatcher. If, on startup, a filter has passed its expiry date (somewhat like a 'best before' date, I suppose), then it is loaded for this session, but shown with a red 'watch' on the icon: . When you quit this session, this filter will not be saved to the filters file, so if you want to keep it, you will have to edit it and change the expiry date.

To warn you that certain filters have expired, you can tell set a preference to have MT-NewsWatcher put up a dialog on startup that says how many filters have expired. This behaviour is controlled by a checkbox in the 'Filtering options' preferences panel, which is labelled as 'Notify about expired filters on startup'.

More filter settings In the lower right hand side of the filter edit dialog, there are some other controls to prevent this filter being overriden by others, and to temporarily disable this filter. Overriding is described below.

The disabling option is useful to 'turn off' a filter temporarily without actually deleting it. You might want to do this, for example, if the filter requires that extra headers be downloaded from the server, and you are on a slow connection so can't tolerate this extra time. You can also disable filters without going through this dialog by option-double-clicking filters in the Filters Window.

Preventing overriding Filters are applied to articles in the order in which they appear in the Filters Window, from the global filters (in the '*' group) to the group-specific ones, and in order within each group category. However, some articles may be matched by more than one filter (say a global filter that kills articles with "cash" in the subject line, but a group-specific filter that hilites articles with "Power Mac" in the subject; the article's subject may read: "Good cash wanted for Power Mac"). Normally, the last (most specific) filter applied is the one which has effect for the article. However, you can ensure that a filter which is not the last to be applied is not overridden by checking the 'Don't override this filter' button. In the above example, the article would then be killed by the first filter.

Go back to the page on filtering, or read the page on using regular expressions for string matching.

MT-NewsWatcher
Download Basics Speech Recognition Filtering Cool Features