> This page location: /docs.md > /docs/tools-and-connectors.md > /docs/spark-connector/current.md > /docs/spark-connector/current/batch-mode.md

> For the complete MongoDB documentation index, see www.mongodb.com/docs/llms.txt

<!--
Tab options on this page. Append to the .md URL to filter:
  ?tabs=<id,...>   select specific tabs (e.g. ?tabs=nodejs,shell)
  ?allTabs=true    include every tab
  (no param)       default: one tab per tabset

Available tabs:
  drivers: java-sync, python, scala
-->

# Read from MongoDB in Batch Mode

## Overview

### Java (Sync)

To read data from MongoDB, call the `read()` method on your `SparkSession` object. This method returns a `DataFrameReader` object, which you can use to specify the format and other configuration settings for your batch read operation.

You must specify the following configuration settings to read from MongoDB:

| Setting | Description |
| --- | --- |
| `dataFrame.read.format()` | Specifies the format of the underlying input data source. Use `mongodb` to read from MongoDB. |
| `dataFrame.read.option()` | Use the `option` method to configure batch read settings, including the MongoDB deployment [connection string](https://www.mongodb.com/docs/manual/reference/connection-string/), MongoDB database and collection, and partitioner configuration. For a list of batch read configuration options, see the [Batch Read Configuration Options](https://www.mongodb.com/docs/spark-connector/current/batch-mode/batch-read-config.md#std-label-spark-batch-read-conf) guide. |

The following code example shows how to use the previous configuration settings to read data from `people.contacts` in MongoDB:

```java
Dataset<Row> dataFrame = spark.read()
                       .format("mongodb")
                       .option("database", "people")
                       .option("collection", "contacts")
                       .load();
```

**Tip: DataFrame Type**

`DataFrame` doesn't exist as a class in the Java API. Use `Dataset<Row>` to reference a DataFrame.

## Schema Inference

### Java (Sync)

When you load a Dataset or DataFrame without a schema, Spark samples the records to infer the schema of the collection.

Suppose that the MongoDB collection `people.contacts` contains the following documents:

```javascript
{ "_id" : ObjectId("585024d558bef808ed84fc3e"), "name" : "Bilbo Baggins", "age" : 50 }
{ "_id" : ObjectId("585024d558bef808ed84fc3f"), "name" : "Gandalf", "age" : 1000 }
{ "_id" : ObjectId("585024d558bef808ed84fc40"), "name" : "Thorin", "age" : 195 }
{ "_id" : ObjectId("585024d558bef808ed84fc41"), "name" : "Balin", "age" : 178 }
{ "_id" : ObjectId("585024d558bef808ed84fc42"), "name" : "Kíli", "age" : 77 }
{ "_id" : ObjectId("585024d558bef808ed84fc43"), "name" : "Dwalin", "age" : 169 }
{ "_id" : ObjectId("585024d558bef808ed84fc44"), "name" : "Óin", "age" : 167 }
{ "_id" : ObjectId("585024d558bef808ed84fc45"), "name" : "Glóin", "age" : 158 }
{ "_id" : ObjectId("585024d558bef808ed84fc46"), "name" : "Fíli", "age" : 82 }
{ "_id" : ObjectId("585024d558bef808ed84fc47"), "name" : "Bombur" }
```

The following operation loads data from `people.contacts` and infers the schema of the DataFrame:

```java
Dataset<Row> dataFrame = spark.read()
                              .format("mongodb")
                              .option("database", "people")
                              .option("collection", "contacts")
                              .load();
```

To see the inferred schema, use the `printSchema()` method on your `Dataset<Row>` object, as shown in the following example:

```java
dataFrame.printSchema();
```

**Output:**

```none
root
 |-- _id: struct (nullable = true)
 |    |-- oid: string (nullable = true)
 |-- age: integer (nullable = true)
 |-- name: string (nullable = true)
```

To see the data in the DataFrame, use the `show()` method on your `DataFrame` object, as shown in the following example:

```java
dataFrame.show();
```

**Output:**

```none
{ "_id" : ObjectId("585024d558bef808ed84fc3e"), "name" : "Bilbo Baggins", "age" : 50 }
{ "_id" : ObjectId("585024d558bef808ed84fc3f"), "name" : "Gandalf", "age" : 1000 }
{ "_id" : ObjectId("585024d558bef808ed84fc40"), "name" : "Thorin", "age" : 195 }
{ "_id" : ObjectId("585024d558bef808ed84fc41"), "name" : "Balin", "age" : 178 }
{ "_id" : ObjectId("585024d558bef808ed84fc42"), "name" : "Kíli", "age" : 77 }
{ "_id" : ObjectId("585024d558bef808ed84fc43"), "name" : "Dwalin", "age" : 169 }
{ "_id" : ObjectId("585024d558bef808ed84fc44"), "name" : "Óin", "age" : 167 }
{ "_id" : ObjectId("585024d558bef808ed84fc45"), "name" : "Glóin", "age" : 158 }
{ "_id" : ObjectId("585024d558bef808ed84fc46"), "name" : "Fíli", "age" : 82 }
{ "_id" : ObjectId("585024d558bef808ed84fc47"), "name" : "Bombur" }
```

### Specify Known Fields with Schema Hints

You can specify a schema containing known field values to use during schema inference by specifying the `schemaHints` configuration option. You can specify the `schemaHints` option in any of the following Spark formats:

| Type | Format |
| --- | --- |
| DDL | `<field one name> <FIELD ONE TYPE>, <field two name> <FIELD TWO TYPE>` |
| SQL DDL | `STRUCT<<field one name>: <FIELD ONE TYPE>, <field two name>: <FIELD TWO TYPE>` |
| JSON | `{ "type": "struct", "fields": [
{ "name": "<field name>", "type": "<field type>", "nullable": <true/false> },
{ "name": "<field name>", "type": "<field type>", "nullable": <true/false> }]}` |

The following example shows how to specify the `schemaHints` option in each format by using the Spark shell. The example specifies a string-valued field named `"value"` and an integer-valued field named `"count"`.

```scala
import org.apache.spark.sql.types._

val mySchema = StructType(Seq(
    StructField("value", StringType),
    StructField("count", IntegerType))

// Generate DDL format
mySchema.toDDL

// Generate SQL DDL format
mySchema.sql

// Generate Simple String DDL format
mySchema.simpleString

// Generate JSON format
mySchema.json
```

You can also specify the `schemaHints` option in the Simple String DDL format, or in JSON format by using PySpark, as shown in the following example:

```python
from pyspark.sql.types import StructType, StructField, StringType, IntegerType

mySchema = StructType([
   StructField('value', StringType(), True),
   StructField('count', IntegerType(), True)])

# Generate Simple String DDL format
mySchema.simpleString()

# Generate JSON format
mySchema.json()
```

## Filters

### Java (Sync)

When using filters with DataFrames or Datasets, the underlying MongoDB Connector code constructs an [aggregation pipeline](https://www.mongodb.com/docs/manual/core/aggregation-pipeline/) to filter the data in MongoDB before sending it to Spark. This improves Spark performance by retrieving and processing only the data you need.

MongoDB Spark Connector turns the following filters into aggregation pipeline stages:

- And

- EqualNullSafe

- EqualTo

- GreaterThan

- GreaterThanOrEqual

- In

- IsNull

- LessThan

- LessThanOrEqual

- Not

- Or

- StringContains

- StringEndsWith

- StringStartsWith

You can use [Java Aggregation Expressions](https://www.mongodb.com/docs/drivers/java/sync/upcoming/fundamentals/aggregation-expression-operations/) to filter your data.

Consider a collection named `fruit` that contains the following documents:

```javascript
{ "_id" : 1, "type" : "apple", "qty" : 5 }
{ "_id" : 2, "type" : "orange", "qty" : 10 }
{ "_id" : 3, "type" : "banana", "qty" : 15 }
```

First, create a DataFrame to connect to your default MongoDB data source:

```java
Dataset<Row> df = spark.read()
                       .format("mongodb")
                       .option("database", "food")
                       .option("collection", "fruit")
                       .load();
```

The following example retrieves only records in which the value of `qty` field is greater than or equal to `10`:

```java
df.filter(df.col("qty").gte(10))
```

The operation outputs the following:

```none
{ "_id" : 2, "qty" : 10.0, "type" : "orange" }
{ "_id" : 3, "qty" : 15.0, "type" : "banana" }
```

## SQL Queries

### Java (Sync)

Before running SQL queries on your Dataset, you must register a temporary view for the Dataset.

The following operation registers a `characters` table and then queries it to find all characters that are 100 or older:

```java
implicitDS.createOrReplaceTempView("characters");
Dataset<Row> centenarians = spark.sql("SELECT name, age FROM characters WHERE age >= 100");
centenarians.show();
```

`centenarians.show()` outputs the following:

```sh
{ "name" : "Gandalf", "age" : 1000 }
{ "name" : "Thorin", "age" : 195 }
{ "name" : "Balin", "age" : 178 }
{ "name" : "Dwalin", "age" : 169 }
{ "name" : "Óin", "age" : 167 }
{ "name" : "Glóin", "age" : 158 }
```

## API Documentation

To learn more about the types used in these examples, see the following Apache Spark API documentation:

### Java (Sync)

- [Dataset\<T>](https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/Dataset.html)

- [DataFrameReader](https://spark.apache.org/docs/latest/api/java/org/apache/spark/sql/DataFrameReader.html)
