[DOC] Create TraceQL metrics sampling docs

[knylander-grafana]

What this PR does:

Creates documentation for TraceQL metrics sampling (#5469) using AI to evaluate the original PR and make documentation recommendations.

FYI - @mdisibio

Which issue(s) this PR fixes:
Fixes https://github.com/grafana/tempo-squad/issues/838
Related: #5469

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[github-actions]

💻 Deploy preview deleted.

[knylander-grafana]

This was an interesting experiment, however, the things introduced are probably subtly wrong. We might spend more time reviewing this PR than it would take us to actually write the content ourselves. I'm going to close this PR and we can start over.

[mdisibio]

I would agree with what we talked about offline. I think most of this PR seems like filler that we wouldn't want (migration strategies and team workflows?), and other plain incorrect information. Was expecting something much more narrow in subject, like a paragraph or two in the functions page. Perhaps with a different prompt it could do that.

[knylander-grafana]

Reopening this PR to try a test with a modified AI prompt to create the sampling-guide using a more focused prompt. The overall file size now is about 2k words instead of 4k words. The provided examples, migration strategies, and other "best practices" look suspect to me.

I added this statement to the prompt to try and reduce hallucinations: "Think step by step. Consider my question carefully and think of the academic or professional expertise of someone that could best answer my question. You have the experience of someone with expert knowledge in that area. Be helpful and answer in detail while preferring to use information from reputable sources."

[github-actions]

💻 Deploy preview deleted.

[knylander-grafana]

Closing this PR until we decide to update the docs.

[knylander-grafana]

@mdisibio I think we're good for a final review. If it looks good and you approve, we can merge and have this in the 2.9 release.

[tempo-ci-app]

The backport to release-v2.9 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-5595-to-release-v2.9 origin/release-v2.9
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x fba00edbac50d8a2348bbebd068fd53ac0b99dc6

When the conflicts are resolved, stage and commit the changes:

git add . && git cherry-pick --continue

If you have the GitHub CLI installed:

# Push the branch to GitHub:
git push --set-upstream origin backport-5595-to-release-v2.9
# Create the PR body template
PR_BODY=$(gh pr view 5595 --json body --template 'Backport fba00edbac50d8a2348bbebd068fd53ac0b99dc6 from #5595{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title '[release-v2.9] [DOC] Create TraceQL metrics sampling docs' --body-file - --label 'type/docs' --label 'backport' --base release-v2.9 --milestone release-v2.9 --web

Or, if you don't have the GitHub CLI installed (we recommend you install it!):

# Push the branch to GitHub:
git push --set-upstream origin backport-5595-to-release-v2.9

# Create a pull request where the `base` branch is `release-v2.9` and the `compare`/`head` branch is `backport-5595-to-release-v2.9`.

# Remove the local backport branch
git switch main
git branch -D backport-5595-to-release-v2.9