[Spark] Support write partition columns to data files for Delta

This support write partition columns to data files for Delta. The feature is required by Uniform Iceberg as Iceberg spec defines so.

The approach is to copy FileFormatWriter from Spark as a DeltaFileFormatWriter, and add the option and logic there to support writing partition columns.

Closes #2367

GitOrigin-RevId: 77657bb422ce93b924f3cb25548e845477f8632f

Author: lzlfred

spark/src/main/scala/org/apache/spark/sql/delta/DeltaOptions.scala

@@ -289,6 +289,10 @@ object DeltaOptions extends DeltaLogging {
    */
   val STREAMING_SOURCE_TRACKING_ID = "streamingSourceTrackingId"
 
+  /**
+   * An option to control if delta will write partition columns to data files
+   */
+  val WRITE_PARTITION_COLUMNS = "writePartitionColumns"
 
   val validOptionKeys : Set[String] = Set(
     REPLACE_WHERE_OPTION,
@@ -323,7 +327,8 @@ object DeltaOptions extends DeltaLogging {
     "checkpointLocation",
     "path",
     VERSION_AS_OF,
-    TIMESTAMP_AS_OF
+    TIMESTAMP_AS_OF,
+    WRITE_PARTITION_COLUMNS
   )