{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Running Scripts from IPython"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"IPython has a `%%script` cell magic, which lets you run a cell in\n",
"a subprocess of any interpreter on your system, such as: bash, ruby, perl, zsh, R, etc.\n",
"\n",
"It can even be a script of your own, which expects input on stdin."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"import sys"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Basic usage"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To use it, simply pass a path or shell command to the program you want to run on the `%%script` line,\n",
"and the rest of the cell will be run by that script, and stdout/err from the subprocess are captured and displayed."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"hello from Python 2.7.5 (default, Jun 20 2019, 20:27:34) \n",
"[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]\n"
]
}
],
"source": [
"%%script python2\n",
"import sys\n",
"print 'hello from Python %s' % sys.version"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"hello from Python: 3.6.8 (default, Oct 17 2019, 14:24:17) \n",
"[GCC 8.3.0]\n"
]
}
],
"source": [
"%%script python3\n",
"import sys\n",
"print('hello from Python: %s' % sys.version)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"IPython also creates aliases for a few common interpreters, such as bash, ruby, perl, etc.\n",
"\n",
"These are all equivalent to `%%script <name>`"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"#%%ruby\n",
"#puts \"Hello from Ruby #{RUBY_VERSION}\""
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"hello from /usr/bin/bash\n"
]
}
],
"source": [
"%%bash\n",
"echo \"hello from $BASH\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Capturing output"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can also capture stdout/err from these subprocesses into Python variables, instead of letting them go directly to stdout/err"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"hi, stdout\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"hello, stderr\n"
]
}
],
"source": [
"%%bash\n",
"echo \"hi, stdout\"\n",
"echo \"hello, stderr\" >&2\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"%%bash --out output --err error\n",
"echo \"hi, stdout\"\n",
"echo \"hello, stderr\" >&2"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"hello, stderr\n",
"\n",
"hi, stdout\n",
"\n"
]
}
],
"source": [
"print(error)\n",
"print(output)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Background Scripts"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"These scripts can be run in the background, by adding the `--bg` flag.\n",
"\n",
"When you do this, output is discarded unless you use the `--out/err`\n",
"flags to store output as above."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"#%%ruby --bg --out ruby_lines\n",
"#for n in 1...10\n",
"# sleep 1\n",
"# puts \"line #{n}\"\n",
"# STDOUT.flush\n",
"#end"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"When you do store output of a background thread, these are the stdout/err *pipes*,\n",
"rather than the text of the output."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"#ruby_lines"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"#print(ruby_lines.read())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Arguments to subcommand"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can pass arguments the subcommand as well,\n",
"such as this example instructing Python to use integer division from Python 3:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.333333333333\n"
]
}
],
"source": [
"%%script python2 -Qnew\n",
"print 1/3"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can really specify *any* program for `%%script`,\n",
"for instance here is a 'program' that echos the lines of stdin, with delays between each line."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"%%script --bg --out bashout bash -c \"while read line; do echo $line; sleep 1; done\"\n",
"line 1\n",
"line 2\n",
"line 3\n",
"line 4\n",
"line 5\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remember, since the output of a background script is just the stdout pipe,\n",
"you can read it as lines become available:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
}
},
"outputs": [],
"source": [
"import time\n",
"tic = time.time()\n",
"line = True\n",
"while True:\n",
" line = bashout.readline()\n",
" if not line:\n",
" break\n",
" sys.stdout.write(\"%.1fs: %s\" %(time.time()-tic, line))\n",
" sys.stdout.flush()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuring the default ScriptMagics"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The list of aliased script magics is configurable.\n",
"\n",
"The default is to pick from a few common interpreters, and use them if found, but you can specify your own in ipython_config.py:\n",
"\n",
" c.ScriptMagics.scripts = ['R', 'pypy', 'myprogram']\n",
"\n",
"And if any of these programs do not apear on your default PATH, then you would also need to specify their location with:\n",
"\n",
" c.ScriptMagics.script_paths = {'myprogram': '/opt/path/to/myprogram'}"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.8"
}
},
"nbformat": 4,
"nbformat_minor": 4
}