This module will introduce you to the benefits of using shell commands to perform data management and analysis as part of your workflow. The shell offers a powerful and succint way of accessing the core functionality of your computer system, and can be used to automate data processing. You interact with the shell via the command line, typing out tasks you want your computer to perform.
- Navigate through a file system and list files within different folders using the commmand line
- Create, move, and delete files using the command line
- Run programs and open files from the command line
- Combine commands, such as filters, using pipes to perform more complex data processing tasks
- Reason about how to automate simple tasks using the command line.
File System Navigation
Starting from any directory, you are able to navigate efficiently to and from any other directory.
Creating, Moving and Deleting
Using the shell, you are able to create, move and delete files and directories.
Born to Shell
Create a shell script that takes a file name as a variable, performs some calculations on the file data, such as number of rows, and prints the result to the console.
Pipes and Filters
Given a folder with a set of files, you are able to find all files of a certain type, i.e “.dat”,filter those files based on a criteria such as the number of lines in a file, and write the resultng list of files into a text file.
Create a shell script that uses a regular expression to find all files named using a certain pattern, rename those files using a different pattern, and save the list of changed files to a text file.
Introduction to the Command Line
An introduction to the command line - how to navigate directories and create, move, and delete files
- Video: Introducing Ubuntu
- Activity: Introducing the Shell (Software Carpentry)
- Video: Directory Navigation
- Activity: Files and Directories (Software Carpentry)
- Video: Create and Delete Files and Dircetories
- Activity: Creating Things (Software Carpentry)
Data Processing and Automation
How to automate the processing of data with the shell.