Wednesday 19 October 2011

Combined Notes and File System searching

To provide a combined Notes & File System "Desktop" search, allowing users to search their Mail, Archives and local PC at the same time as other data such as Notes Apps, Document Libraries, mapped File Systems and Quickr, the FT Search Manager utilises a separate Notes database, the File Indexer database.

This File Indexer database contains various Java agents, which trawl the defined file systems, either server-based or locally, then uses the Apache open-source POI libraries (http://poi.apache.org and http://pdfbox.apache.org) to extract text from the specified files, add it to dedicated Notes storage databases, and Full-Text index these. So far, it indexes file types doc, docx, xls, xlsx, ppt, pptx, vsd, pdf, txt, htm and html.

These Notes based storage databases are reasonably small (think of them as part of the index) and added automatically to the FT Search Manager. The result allows a Full Text search of the File System contents, including field-level searches on the meta-data, and also use of custom operators, such as "near" ("java near 20 apache" would find documents where 'java' appears within 20 words of 'apache').

Clicking a result opens the file in it's native application, not in Lotus Notes (although if searching via Browser and the Browser supports the application, it will open in a Browser window).

This makes it easy to locate (and process) information, regardless of it's actual location, and even allows iPad, iPhone and other Smartphone users to search File Systems at the same time as Notes-based data.

Here's an iPad user running a search. The various search filters such as 'Person Contains' can be mapped to meta-data from each data source, shown/hidden etc.








Searching is also available from various integration points in the Notes Client, Browser, iNotes or Web Service, including Discovery/Compliance searching (Admin, Power User or User) and Notes & Web Portal searching. 

For example, you could use the Web Service to search server-based File Systems, returning results as XML and bypassing the Notes/Domino UI altogether.

For more detail, demonstrations or an obligation-free trial of the Search Manager, please visit http://www.ionetsoftware.com/search.

For a copy of the File Indexer database, please contact us via http://www.ionetsoftware.com/contact

Wednesday 12 October 2011

Archiver for Notes - Automatic Updates

One of the features that gets most use in our Archiver for Notes and Search Manager products is automatic updating. This is an optional setting that can be used to auto-update the products in a customer environment, most usually to add bugfixes.

Via this procedure, we get notification of errors that occur during archiving (and when archiving millions of documents in remote environments, especially in custom applications, there can be document-related errors).

We then investigate the problem, and if necessary provide a patch for it. This patch can then be automatically applied in the remote environment, and the customer notified as to what was fixed, and why. Alternatively the customer can instead be notified there is a patch available, and download/apply it manually.

Basically the process itself is as follows;

1. Each night, the Archiver contacts our website via a Java agent and checks for a flag indicating there is a newer release available than it is already. If it finds a newer release, it will either;

a) email a listed email address there is a new version to download, the reason for the update, and provide a url to the new template, 

OR 

b) download the new template to a designated location, run fixup on it using a Remote Console call, then create an AdminP request to sign it using the Server ID. The Designer task then updates the design of the production database in the normal way, using the newly downloaded template. 

A second agent then verifies that the new design has been applied correctly, re-enables any agents that were disabled as a result of the update, and advises the listed email address that the design was updated, and for what reason.

Of course we need to manage the connection out of the customers environment (proxies, security etc) and amend the Java security policy on the server to allow our agent to set system properties during execution.

We find this process works very well, and is a useful way to reduce the demands on customers, even when the problem stems from dodgy data in their own environment.

For more details, please contact info@ionetsupport.com.