Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/import/import-files/generic-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,17 @@ As of Flux 1.3.0, text can be extracted from files via [Apache Tika](https://tik
documents in MarkLogic. This is typically useful when importing binary content such as PDF and Word files, where both
the binary file and extracted text can be stored in MarkLogic.

### Behavior change in Flux 2.1.2: Microsoft Office file extraction

Flux 2.1.2 upgrades Apache Tika from 3.3.1 to 3.3.2. Tika 3.3.2 changes the default parser for Microsoft Office OOXML
files (`.docx`, `.pptx`, `.xlsx`, `.vsdx`) from a DOM-based extractor to a SAX-based extractor. The SAX parser is
faster and more memory-efficient, but may produce slightly different whitespace in extracted text compared to previous
Flux versions — for example, paragraph separators may differ.

If your application depends on the exact text output from Office files and you need the previous DOM-based behavior,
you can restore it by providing a [Tika configuration file](https://tika.apache.org/3.3.2/configuring.html) that
sets `useSAXDocxExtractor` and/or `useSAXPptxExtractor` to `false` on the `OfficeParserConfig`.

Text extraction is enabled by including the following option when executing the `import-files` command:

--extract-text
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (c) 2024-2025 Progress Software Corporation and/or its subsidiaries or affiliates. All Rights Reserved.
* Copyright (c) 2024-2026 Progress Software Corporation and/or its subsidiaries or affiliates. All Rights Reserved.
*/
package com.marklogic.flux.impl.importdata;

Expand Down Expand Up @@ -55,7 +55,7 @@ void microsoftFile() {
assertCollectionSize(collection, 2);
JsonNode doc = readJsonDocument("/extraction-files/hello-world.docx-extracted-text.json", collection);
assertEquals("/extraction-files/hello-world.docx", doc.get("source-uri").asText());
assertEquals("Hello world.\n\nThis file is used for testing text extraction.\n", doc.get("content").asText());
assertEquals("Hello world.\nThis file is used for testing text extraction.\n", doc.get("content").asText());
assertEquals("application/vnd.openxmlformats-officedocument.wordprocessingml.document",
doc.get("extracted-metadata").get("Content-Type").asText());
}
Expand Down
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ awssdkVersion=2.35.4
nettyVersion=4.2.16.Final
janinoVersion=3.1.12
langchain4jVersion=1.17.2
tikaVersion=3.3.1
tikaVersion=3.3.2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be done in the Spark connector too since it depends on the core Tika library.


# Define these on the command line to publish to OSSRH
# See https://central.sonatype.org/publish/publish-gradle/#credentials for more information
Expand Down
Loading