I am writing to connector from on prem MongoDB to Bigquery. I have a very difficult point. I convert one binary using decodeBinarytouuid method in java but data have lots of Binary data and also this connector is not specific to one team. So that I need a help for this situation.
Also I use Uuid Representation in MongoClientSettings but it isn’t work.
Hi @Emin_Can_OGUZ and welcome in the MongoDB Community !
What’s in your MongoDB collection initially? Can you share one document maybe from mongosh? Are these _id fields really binary data or it’s actually ObjectIds?
I try also CodecRegistry but it isn’t works I want to try Convert all Binary Data to String.
I try also decodeBinaryToUuid() function in Java. It is works but this function is only convert one _id field. But I want to convert all Binary Data without giving fields.
Well that’s very unusual.
It’s the first time that I actually see a BinData used as an _id. Usually we have ObjectIds which can be represented by an hexadecimal number.
What makes you think that this binary data can be represented as a string?
I have no idea how you could solve this problem really.
Because of I have a connector from on-prem mongo to BQ and I write this code using Java for my company. This connector is not supporting Binary Data. So that I am very trouble.
We are trouble for this situation and we are trying several ways to changing Binary data to String (Uuid is enough for us but this type must be string). Can you help us?
Sorry about my mention but we are so trouble for this problem.
Then all documents that you query for will be of type BsonDocument, and all binary values will be of type BsonBinary. From there you can get the raw byte array and convert it to string using any encoding you want, e.g. java.util.Base64. If you need to also encode the binary subtype, you’ll have to figure out a way to include the subtype as well.
UUIDs can be tricky though, so be careful with how you treat those bytes once you store them. See specifications/uuid.rst at master · mongodb/specifications · GitHub for the gory details. If you are able to start with fresh data, use STANDARD UuidRepresentation for all UUIDs. If you can’t do that, but you can assume JAVA_LEGACY, you might want to convert those to the standard representation before then converting to a String. I can help you with that as well if you need it.
We solve problem in JSONObject. We can’t change default Mongo data. We write new function using BinaryConverter for JsonWriterSettings. This code this like that.
We solved this problem. Thanks for responding. So we are so discussing about adding this feature to driver with PR. We want to PR but we don’t know the PR rules in MongoDB. So that can you advice to PR the this feature or this problem is solved in forum. Which one do you prefer?
You can write a BinaryConverter class service in Java. Because of that JsonBinaryConverter is not here in Mongo. So that you can write a class after than you called the class new statement.
I want PR but I haven’t got any free time so that I share my code.
import org.bson.BsonBinary;
import org.bson.UuidRepresentation;
import org.bson.internal.UuidHelper;
import org.bson.json.Converter;
import org.bson.json.StrictJsonWriter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class JsonBinaryConverter implements Converter<BsonBinary> {
private static final Logger LOGGER = LoggerFactory.getLogger(JsonDateTimeConverter.class);
@Override
public void convert(BsonBinary value, StrictJsonWriter writer) {
try {
writer.writeString(UuidHelper.decodeBinaryToUuid(value.getData(),value.getType(), UuidRepresentation.JAVA_LEGACY).toString());
} catch (Exception e) {
LOGGER.info(String.format("Fail to convert offset %d to JSON date",value),e);
}
}
}
Also if you have a time value you must convert this like JsonDateTimeConverter()
import org.bson.json.Converter;
import org.bson.json.StrictJsonWriter;
import java.time.Instant;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.Date;
public class JsonDateTimeConverter implements Converter<Long> {
static final DateTimeFormatter DATE_TIME_FORMATTER = DateTimeFormatter.ISO_INSTANT
.withZone(ZoneId.of("UTC"));
@Override
public void convert(Long value, StrictJsonWriter writer) {
try {
Instant instant = new Date(value).toInstant();
String s = DATE_TIME_FORMATTER.format(instant);
writer.writeString(s);
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
You will call this service in your main statement. If you have a problem I will help you
I think I fixed the issue by converting BSON byte[] to string and returned to JSON Writer.
here is the code snippet
@Override
public void convert(BsonBinary value, StrictJsonWriter writer) {
System.out.println("SUBTYPE"+value.getType());
int sg= value.getData().length;
System.out.println("LENGHTH"+sg);
byte[] d = value.getData();
String s = new String(d);
System.out.println("DATA-->"+s);
try {
writer.writeString(s);
System.out.println(writer);
} catch (Exception e) {
System.out.println("EXCEPTION"+e);
// LOGGER.info(String.format("Fail to convert offset %s to JSON date",value),e);
}
JSON Response with decoded BINARY:
“Payload”: “{\r\n “postId”: 1,\r\n “id”: 1,\r\n “name”: “id labore ex et quam laborum”,\r\n “email”: “Eliseo@gardner.biz”,\r\n “body”: “laudantium enim quasi est quidem magnam voluptate ipsam eos\r\ntempora quo necessitatibus\r\ndolor quam autem quasi\r\nreiciendis et nam sapiente accusantium”\r\n"path” :"\game\forum\files\index.php$!@#%^&*()"\r\n }",
@Emin_Can_OGUZ Could you please help how to remove \r\n \ special characters in JSON response.
In my IDE console printing my payload response with no characters like \r\n
as shown below.