Index Manager Message: Partial indexing only. Exceeded memory limit.

Question

I have thousands of small pdf files (under 5 mb) in an index where IM has message: "Partial indexing only. Exceeded memory limit."

Index size: 1139.54 MB
Index Count: 64,288

In Configuration, Memory Manager Settings, I have disabled/enabled Limit Memory Consumption, then refreshed index, but still get the same message.

What does this message mean?
How do I achieve full indexing for these files?

using FLP Version 2022 - Build 3283

Answer 1

The false labelling of files as 'Partial indexing only' was a bug in that version of the program. It was fixed in Build 3286.

If you're still getting the warning then the source documents contain too many terms for the default memory settings.

Indexing memory usage

During the indexing of a document the indexer maintains a list of unique terms within the document, therefore lots of unique terms requires lots of memory. Internally the indexer has four different memory settings:

• 1: Very high
• 2: High
• 3: Normal (default)
• 4: Low

The setting determines the maximum amount of RAM, as a percentage of total RAM, the indexer can use.

At present the only way to change these settings is via the config_v9.xml file, located in the Configuration -> Folders -> Config folder. To increase the RAM usage to maximum, add/modify the IndexMemMgr section and set the value to '1' for both MaxRamUsageType and MaxRamPerDocType :

<cfg ver="2">
    <section name="IndexMemMgr">
        <MaxRamUsageType n="1" />
        <MaxRamPerDocType n="1" />
    </section>
...

Indexing threads

Since the indexer can concurrently index documents on separate threads the indexer may throttle the number of threads if it decides the host system's RAM could potentially be exhausted.