spark conda environment yml file containing packages that don't exist

[thomasaarholt]

Hi!
This spark conda environment file specifies to download azureml-synapse==0.0.1 with pip. This version doesn't exist (and if it did then it would be terribly old, as the latest version is 1.59).

I'm following these docs, which include (under "Step 4: Run the script provided"), instructions to run this script, which in turn uses the above conda environment when creating a spark 3.4 environment.

That script fails when trying to install azureml-synapse==0.0.1, because it does not exist.

I'd really appreciate it if you could provide a functioning conda environment. I see that there is different one for 3.3 at Azure-Samples/Synapse, but that one won't work for spark 3.4.

[thomasaarholt]

@SparkVHDReleaseManagementTeam, I just noticed that you edited the file 13 hours ago (I started writing this issue yesterday, before your edit). Is there something I'm missing when trying to create an environment? I don't understand how this is (apparently?) working for you?