Skip to content

Commit 606d306

Browse files
authored
docs: add DataFusion security guidance (#26064)
## Which issue does this PR close? - Closes #26061. ## Rationale for this change Applications embedding DataFusion may accept SQL from users, but the library guide does not explain security-relevant defaults or how available controls contribute to hardening. ## What changes are included in this PR? Adds a library guide covering SQL statement restrictions, opt-in local-file access, query memory limits, spill storage location and size limits, and the authorization and isolation responsibilities of the hosting application. ## What is the testing strategy for this PR? - `./ci/scripts/doc_prettier_check.sh` (passed after the documentation update). ## Are there any user-facing changes? Adds a “Securing DataFusion” page to the Library User Guide and links it from the guide and documentation navigation. The page now explains that memory limits do not cap spill disk use and names the runtime settings for controlling temporary storage. AI assistance was used in preparing this contribution. --------- Signed-off-by: Efe Gökdemir <gokdemirefe1903@gmail.com>
1 parent 447368e commit 606d306

3 files changed

Lines changed: 74 additions & 0 deletions

File tree

‎docs/source/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,7 @@ To get started, see
137137
:caption: Library User Guide
138138

139139
library-user-guide/index
140+
library-user-guide/securing-datafusion
140141
library-user-guide/upgrading/index
141142
library-user-guide/extensions
142143
library-user-guide/using-the-sql-api

‎docs/source/library-user-guide/index.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ for details on how to contribute to DataFusion.
2929
If you haven't reviewed the [architecture section in the docs][docs], it's a
3030
useful place to get the lay of the land before starting down a specific path.
3131

32+
For guidance on running DataFusion with SQL from untrusted users, see
33+
[Securing DataFusion](securing-datafusion.md).
34+
3235
DataFusion is designed to be extensible at all points, including
3336

3437
- [x] User Defined Functions (UDFs)
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
<!---
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->
19+
20+
# Securing DataFusion
21+
22+
As described in the [DataFusion security policy](https://github.com/apache/datafusion/blob/main/SECURITY.md),
23+
the end application is responsible for security decisions. The settings below
24+
can help control what a query can do.
25+
26+
## Restrict SQL statements
27+
28+
[`SQLOptions`] allows an application to reject classes of SQL statements when
29+
creating a `DataFrame`. DDL (such as [`CREATE TABLE`](../user-guide/sql/ddl.md))
30+
and DML (such as [`INSERT`](../user-guide/sql/dml.md)) are allowed by default.
31+
Disable the classes that the application does not need and pass the options to
32+
[`SessionContext::sql_with_options`] for every user-provided query:
33+
34+
```rust
35+
use datafusion::prelude::*;
36+
37+
let options = SQLOptions::new()
38+
.with_allow_ddl(false)
39+
.with_allow_dml(false)
40+
.with_allow_statements(false);
41+
42+
let dataframe = ctx.sql_with_options(sql, options).await?;
43+
```
44+
45+
## Limit file access
46+
47+
[`SessionContext::enable_url_table()`] is an opt-in feature that lets SQL query
48+
local files by path. Leave it disabled when users should only query tables
49+
registered by the application. If it is needed, run DataFusion with filesystem
50+
permissions limited to the files the application intends to expose.
51+
52+
## Set query memory limits
53+
54+
Set an appropriate `datafusion.runtime.memory_limit` for the workload using the
55+
[runtime configuration settings](../user-guide/configs.md#runtime-configuration-settings).
56+
This limits memory used by DataFusion's query execution memory pool; use
57+
process- or container-level resource limits as well when a hard bound on total
58+
application memory is required.
59+
60+
## Bound spill storage
61+
62+
When an execution operator supports spilling, DataFusion may write intermediate
63+
query data to temporary files under memory pressure. Use
64+
`datafusion.runtime.temp_directory` and
65+
`datafusion.runtime.max_temp_directory_size` to configure the location and size
66+
of that temporary storage; see the [runtime configuration settings](../user-guide/configs.md#runtime-configuration-settings).
67+
68+
[sqloptions]: https://docs.rs/datafusion/latest/datafusion/execution/context/struct.SQLOptions.html
69+
[sessioncontext::sql_with_options]: https://docs.rs/datafusion/latest/datafusion/execution/context/struct.SessionContext.html#method.sql_with_options
70+
[sessioncontext::enable_url_table()]: https://docs.rs/datafusion/latest/datafusion/execution/context/struct.SessionContext.html#method.enable_url_table

0 commit comments

Comments
 (0)