Execute shell commands as flows

Learn how to use the Prefect CLI for executing shell commands as flows.

This page explains how to use the watch and serve Prefect CLI commands to execute and schedule shell commands as Prefect flows, including how to:

• run a shell command as a Prefect flow on-demand with watch
• schedule a shell command as a recurring Prefect flow using serve

Prerequisites

Before you begin, ensure you have:

• A basic understanding of Prefect flows. Start with the Getting started guide if necessary.
• A recent version of Prefect installed in your command line environment. Follow these instructions if you have any issues.

The watch command

The watch command wraps any shell command in a Prefect flow for instant execution. This is ideal for quick tasks or integrating shell scripts into your workflows.

Example usage

To fetch the current weather in Chicago using the curl command, use the following Prefect CLI command:

prefect shell watch "curl http://wttr.in/Chicago?format=3"

This command makes a request to wttr.in, a console-oriented weather service, and prints the weather conditions for Chicago.

Benefits of watch

• Immediate feedback: Execute shell commands within the Prefect framework for immediate results.
• Easy integration: Blend external scripts or data fetching into your data workflows.
• Visibility and logging: Use Prefect’s logging to track the execution and output of your shell tasks.

Deploy with serve

To run shell commands on a schedule, the serve command creates a Prefect deployment for regular execution. This is a quick way to create a deployment served by Prefect.

Example usage

To set up a daily weather report for Chicago at 9 AM, use the serve command as follows:

prefect shell serve "curl http://wttr.in/Chicago?format=3" --flow-name "Daily Chicago Weather Report" --cron-schedule "0 9 * * *" --deployment-name "Chicago Weather"

This command schedules a Prefect flow to fetch Chicago’s weather conditions daily, providing consistent updates without manual intervention. To manually fetch the Chicago weather, create a run of your new deployment from the UI or CLI.

To shut down your server and pause your scheduled runs, hit ctrl + c in the CLI.

Benefits of serve

• Automated scheduling: Schedule shell commands to run automatically, ensuring critical updates are generated and available on time.
• Centralized workflow management: Manage and monitor your scheduled shell commands inside Prefect for a unified workflow overview.
• Configurable execution: Customize execution frequency, concurrency limits, and other parameters to suit your project’s needs and resources.