ParquetWriter write method is taking too long to write single avro message

[aksamit26]

When i am reading data form Kafka , i am deserialising it to Avro record. But ParquetWriter write method is taking around 2~5 secs to write one single avro record.
Is there any such issue surfaced earlier. What can be possible solve for this ?

[HenryCaiHaiying]

You can do a few thread dump to see what the process stuck at. Adding memory probably will help parquet conversion.

…

On Tue, Jul 27, 2021 at 1:52 AM aksamit26 ***@***.***> wrote: When i am reading data form Kafka , i am deserialising it to Avro record. But ParquetWriter write method is taking around 2~5 secs to write one single avro record. Is there any such issue surfaced earlier. What can be possible solve for this ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2133>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABYJP74TKFG6EMN5TRTVTJDTZZXVVANCNFSM5BBYSN7A> .