Shuffle+Repartition on an RDD could lead to incorrect answers.Support sending messages over 2GB from memory. Support replicating blocks larger than 2 GB.Nested schema pruning for Parquet tables.Cache the function name from the external catalog for lookupFunctions.Reference resolution for large number of columns should be faster.Implement EXCEPT ALL and INTERSECT ALL.Support column resolution of fully qualified column name.Coalesce and Repartition Hint for SQL Queries.Built-in Avro data source: Inline Spark-Avro package with logical type support, better performance and usability.Higher-order functions: Add a lot of new built-in functions, including higher-order functions, to deal with complex data types easier.Now you can build Spark with Scala 2.12 and write Spark applications in Scala 2.12. Scala 2.12 Support: Add experimental Scala 2.12 support.Barrier Execution Mode: Support Barrier Execution Mode in the scheduler, to better integrate with deep learning frameworks.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |