Skip to content

Rr 115 update docs - #138

Open
ryanraaschCDC wants to merge 21 commits into
mainfrom
rr-115-update-docs
Open

Rr 115 update docs#138
ryanraaschCDC wants to merge 21 commits into
mainfrom
rr-115-update-docs

Conversation

@ryanraaschCDC

Copy link
Copy Markdown
Collaborator

No description provided.

@ryanraaschCDC
ryanraaschCDC requested a review from Copilot March 19, 2026 13:11
@ryanraaschCDC ryanraaschCDC linked an issue Mar 19, 2026 that may be closed by this pull request

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the documentation to improve onboarding/troubleshooting guidance and reflect newer client capabilities in cfa-cloudops.

Changes:

  • Add new troubleshooting sections for CloudClient instantiation and file-path/mount issues during jobs
  • Expand the CloudClient getting-started notebook with clearer guidance around env files, blob containers, ACR uploads, pools, and jobs
  • Update docs to mention ContainerAppClient and adjust the service-principal sample .env

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
docs/troubleshooting.md Adds new troubleshooting guidance for CloudClient instantiation and file-not-found issues.
docs/overview.md Mentions ContainerAppClient as a repo component.
docs/files/sp_sample.env Removes AZURE_SP_CLIENT_ID from the SP sample env.
docs/examples/getting_started/cloudclient_walkthrough.ipynb Expands the getting-started walkthrough with additional explanations and examples.
docs/examples/getting_started/Dockerfile Replaces the minimal Dockerfile with a more explanatory example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
Comment thread docs/troubleshooting.md Outdated
Comment thread docs/troubleshooting.md Outdated
Comment thread docs/troubleshooting.md Outdated
Comment thread docs/overview.md
Comment thread docs/files/sp_sample.env
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
ryanraaschCDC and others added 5 commits March 19, 2026 06:30
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@ryanraaschCDC
ryanraaschCDC requested a review from sfrosenb March 19, 2026 13:36
@sfrosenb
sfrosenb marked this pull request as ready for review April 16, 2026 14:41

@sfrosenb sfrosenb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cloudops Comments Not Directly Addressed by PR Suggestions

Overview

Several members of my team spent significant time (over 2 weeks) debugging the walkthrough for this package before we could get it to work, and some of us are still encountering issues. This seems to point to some underlying difficulties with CFA–Azure interaction or with the package itself. Below are a few suggestions that may help address these challenges.

Stress‑Test Functions

Several cloudops functions are somewhat brittle, failing under certain assumptions that fall outside typical user workflows.

Cc.monitor_task(download_task_output = True)

  • This function can fail if any tasks associated with a job no longer have nodes.
  • Possible improvements:
    • Would it be possible to add a try–catch or filtering step to ignore tasks lacking nodes?
    • Alternatively, a more informative error message could help users understand when a new job is required.
    • If helpful, I’m happy to submit a feature request.

Cc.create_job

  • If the pool does not exist, the job appears to be created, but tasks submitted to it remain queued indefinitely.

Pre‑Identify User Issues

  • Reviewing feature requests from the predecessor package (cfa-azure) may help proactively address common user concerns.

Improve Error Messages

  • Cc.monitor_task fails when tasks lack associated nodes, but the resulting error is not very informative.

Reduce Debugging Time

Debugging has been more challenging because—even with vmsize = 'xsmall'—the transition from queued → running typically takes at least 4 minutes.

Possible improvements:

  • Is there any possibility of creating a pool that allows for faster debugging?
  • Would dedicated nodes help? Or would they pose too high a cost risk if users forget to delete the pool?
  • If startup time cannot be reduced, making it easier to launch multiple tasks for testing might help:
    • Currently, if one task fails earlier than another, the second call to download_task_output errors out.

Other Suggestions

  • In the walkthrough, it could be more helpful for users if argument names were explicitly specified rather than relying on positional interpretation.

Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb
Comment thread docs/examples/getting_started/Dockerfile
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb
Comment thread docs/troubleshooting.md
@sfrosenb

Copy link
Copy Markdown

I think adding clarity around the dockerfile beyond pointing to the generic docker docs would help users and troubleshooting. Your example dockerfile sets WORKDIR to /app but then in the walkthrough, "app" is not used in create_pool() nor the add_task() nor main.py, and docs/troubleshooting.md does not use it either.

It would be helpful to provide the user an example Dockerfile that works with the walkthrough

@ryanraaschCDC ryanraaschCDC linked an issue Apr 28, 2026 that may be closed by this pull request
Comment thread docs/examples/getting_started/cloudclient_walkthrough.ipynb Outdated
@HPSLU
HPSLU requested a review from sfrosenb June 30, 2026 17:23
@ryanraaschCDC

Copy link
Copy Markdown
Collaborator Author

@sfrosenb I've made several changes related to your comments. Let me know if anything else needs to be changed or added. Thanks!

@ryanraaschCDC ryanraaschCDC linked an issue Jul 24, 2026 that may be closed by this pull request
@ryanraaschCDC
ryanraaschCDC requested a review from HPSLU July 24, 2026 16:39
@ryanraaschCDC

Copy link
Copy Markdown
Collaborator Author

@HPSLU please review the PR when you have the time. Thanks!

Comment thread cfa/cloudops/metaflow/examples/BKDockerfile Outdated
Comment thread cfa/cloudops/metaflow/examples/metaflow.env Outdated
Comment thread cfa/cloudops/metaflow/examples/README.md Outdated
Comment thread cfa/cloudops/metaflow/examples/README.md
@ryanraaschCDC ryanraaschCDC linked an issue Jul 29, 2026 that may be closed by this pull request
Comment thread docs/CloudClient/job.md Outdated

@HPSLU HPSLU left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

identified a few areas needing confirmation to improve clarity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

update docs to Zensical Documentation Updates provide Docker help update documentation

5 participants