|
|
| Line 1: |
Line 1: |
| ''This page is part of [[Programming and CS - Getting Started]]'' | | ''This page is part of [[Programming and CS - Getting Started]]'' |
|
| |
|
| This page is meant to help a new student in a CS course at ISU to get familiar with using Linux (in particular using the text-based terminal to interact with the system).
| | =Git= |
| | [https://en.wikipedia.org/wiki/Git Git] is a version control system that can be used to keep track of your files (programming code, but also other files as well). |
|
| |
|
| =Getting Started=
| | Two of the most popular installations of git (as of 2020) are [https://github.com/ github] and [https://gitlab.com/ gitlab]. When Microsoft purchased github, it became fashionable to prefer gitlab. Github allows for projects to be hosted on their servers. Gitlab allows for hosting either on their servers or to run a local installation of gitlab on ones own servers. |
| # '''Read''' - [https://ubuntu.com/tutorials/command-line-for-beginners#1-overview Ubuntu Linux Command Line Tutorial] or [http://www.ee.surrey.ac.uk/Teaching/Unix/ Linux Tutorial] - this gives you some background and introduces you to some of the terms that we use. After reading through the tutorial you have a better chance of being able to follow along the rest of the steps in this Getting Started.
| |
| # '''System Setup''' - [[Linux - System Setup]] - to get your computer setup (programs installed, etc.).
| |
| # '''Using the Terminal'''
| |
| ## [[Linux Terminal - Files and Directories and Getting Around]]
| |
| ## [[Linux Terminal - System Information]]
| |
| ## [[Linux Terminal - Text Editors]]
| |
| ## [[Using Linux - Large Text Files]]
| |
| ## [[Linux Terminal - More Commands to Know]]
| |
|
| |
|
| =Common Mistakes=
| | Of late, we have hosted projects using gitlab either on gitlab's servers or on ISU's local installation of gitlab. |
| * When logging in remotely to the CS server, when you are typing your password you might not see anything printed on your screen while you type. That is a security feature so that someone watching your screen would not know how many characters are in your password. Just finish typing your password and type enter when finished.
| |
| * Note that all commands and file names in Linux are '''case sensitive'''. hello.txt is different than Hello.txt. The copy command is cp, Cp would not work, and neither would cP or CP.
| |
| * The characters _ and - are different, and having a space in a file name is different than not having one ("hello world.txt" is different than "helloworld.txt").
| |
| * For Linux directories and filenames, the "/" character is used to separate directories. On Windows, the "\" character is used instead. If you typed <pre>cd /home/cs151</pre> then this changes the directory to the cs151 user's home directory. If you instead typed <pre>cd \home\cs151</pre> you would get an error.
| |
| * The "top" or "highest level" of the Linux file system is the directory / and this is comparable to the "C:\" that was traditionally the top of the file system in Windows. In Windows there can be multiple drives (so D:\, E:\, etc.), but in Linux '''everything''' is underneath / (including additional drives, they will show up at a certain location under /).
| |
| * On Linux, ~ is a shortcut for your home directory. User jkinne@isuad.indstate.edu's home directory is in /home/jkinne@isuad.indstate.edu, so if this user is logged in and types <pre>cd ~</pre> then this means to change directories to /home/jkinne@isuad.indstate.edu/. If user jeff types <pre>cd ~</pre> then this means to change directories to jeff's home directory (which happens to be /home/jeff/).
| |
| * Not all sequences of commands are ''idempotent'' - this is a technical term that means that the sequence of commands gives the same results if executed more than once. For example, executing <code>echo "hello world!" >> hello.txt</code> once creates a file hello.txt that has one line in it, but executing again makes the file have two lines. When you see a sequence of commands to run to complete some task (e.g., from an online tutorial) be careful to execute one command after the other, all the way through, stopping after each to make sure you know what that step was supposed to do.
| |
| * Sometimes the precise syntax of commands might depend on which shell program you are using. bash is the default shell program on the CS server, but zsh is commonly used instead. You can see which shell you are running by executing the <code>ps</code> command and seeing which one is in the printed output from that command. For example, the command for displaying your resource limits is <code>ulimit -a</code> in bash and zsh, but is <code>limit</code> in tcsh. When looking up how to do things online, it is good to include your shell in the search terms (e.g., search for - bash get resource limits - rather than just - get resource limits).
| |
|
| |
|
| =More Reading= | | =Account Setup= |
| Read through another tutorial about Linux. Some options include the follow.
| | '''On gitlab.com''' - Account setup on [https://gitlab.com/ gitlab.com] is just like any other website. You create a free account, choose your username/password, etc. The account is yours and is not tied in any particular way to ISU. You can give access to your projects in much the same way as you give access to files on google drive, onedrive, dropbox, etc. |
| * [http://www.linuxcommand.org/ Linux Console Tutorial]
| |
| * [http://www.reallylinux.com/ Information for Beginning Linux Users]
| |
| * [http://cs.indstate.edu/~sbaker/faq/QuickRefCard.pdf Unix/Linux Quick Reference Card]
| |
|
| |
|
| =Self Test=
| | '''On gitlab.indstate.edu''' - ISU's local installation of gitlab is at https://gitlab.indstate.edu/. For this system you use your ISU portal id and password. You do not need to create an account. The first time you login into the system, it initializes an account for you. CS courses and ISU CS projects in general tend to use ISU's local installation because we can tie access based on course rosters, etc. |
| Note: a practice quiz over linux commands and the terminal is here - https://indstate.instructure.com/courses/12565/quizzes/244584
| |
|
| |
|
| Here we give small tasks for you to complete. These are numbered so that we can refer to them. For each you would need to be logged into one of the CS systems (remotely logged into the CS server or using one of the CS lab computers).
| | =Getting Around= |
| | Whether you are using gitlab.com or gitlab.indstate.edu, the functionality and "how to get around" is about the same. You have two main methods for using either. |
|
| |
|
| ''Linux.1'' Log in to the CS server, create a directory named aboutMe in your home directory, and create a text file name.txt. In the name.txt file, give your full name.
| | ==Web browser interface== |
| | You can view projects within your web browser. How to get around is fairly intuitive - as is the case with any web page, you look around and click around to see what different things do. When you have something specific you want to do but don't see how to do, you ask the internet or ask your instructor (and we may update this wiki to give instructions on how to do it). |
|
| |
|
| ''Linux.2'' In your aboutMe directory, create a file courses.txt and list which CS courses you are currently taking or plan to take in the upcoming term.
| | ==Terminal interface== |
| | Git allows for downloading/uploading/etc. files in your projects from the terminal. In order to do this, you need to (a) have a git client installed on your system, (b) understand the terminology that git uses for keeping track of files and the steps in using git to download/update/upload files, (c) make it so. |
|
| |
|
| ''Linux.3'' Log in to the CS server, create a directory named notes in your home directory, and create a text file named linux.txt and put in some links to linux tutorials or information that you have found helpful.
| | Some reading material about using git - https://www.tutorialspoint.com/git/index.htm |
|
| |
|
| ''Linux.4'' Log in to the CS server, create a directory named linux-bootcamp in your home directory.
| | ===Vocab / Cheat Sheet=== |
|
| |
|
| ''Linux.5'' In your linux-bootcamp directory, create a text file shake.txt. In shake.txt, put how many lines are in the shakespeare.txt file mentioned in the information above. | | ===Workflow=== |
| | A common use case is that you will need to download someone else's (e.g., instructor's) gitlab project, create your own version of some part of the project, test/debug your version of the project, eventually put in a request to update the main project with your code. The steps involved in that workflow. |
|
| |
|
| ''Linux.6'' In your shake.txt file, add information about how many times "the" appears in shakespeare.txt. Use the command grep, | | '''Clone''' - To download the current version of the project you (a) browse in your terminal to the directory where you want to have the project code, (b) run <code>git clone url_to_project</code> |
| add to what you had how many lines "the" appears in shakespeare.txt. Hint - use the -c option for grep, also use the -i option so your count will also include "The", "THE", etc. In your shake.txt, you should have the number of lines that contain "the" (including "The", etc.), and also the percentage of the lines that contain the.
| |
|
| |
|
| ''Linux.7'' In your linux-bootcamp directory, create a text file weather.txt. Copy the Indianapolis weather csv file from /var/junk/weather/ into your linux-bootcamp directory. In your weather.txt file, put how many rows of data are in the csv file (use wc, head, and tail). Assuming 365 days per year, put in your weather.txt file how many years of data are in the csv file. | | '''Branch''' - If a branch has already been created for you, then you can set that as your active branch by running <code>git checkout branch_name</code>. If a branch has not yet been created for you, you run <code>git branch branch_name</code> to create a new branch. In either case, you have a branch of the project to work within. |
|
| |
|
| ''Linux.8'' In your weather.txt file, put the maximum PRCP, SNOW, SNWD, TMAX, TMIN, MEAN from the csv file. Hint - use the sort command, and use the options -t, -k, -n, -o. | | '''Do your work''' - You work on your files locally - test, debug, etc. |
|
| |
|
| =Account Configuration=
| | '''Stage/Commit/Push''' - To save your work to the remote server, you need to stage, commit, and push. You can stage/commit batches of changes, or do a stage/commit for each discreet task accomplished. If you are working in some_file.txt, you would run the following sequence of commands. |
| Bash reads configuration information from a file <code>~/.bashrc</code>. A file with good defaults for the CS server are at [https://cs.indstate.edu/~jkinne/training_files/.bashrc .bashrc]. More help on bash - [https://www.gnu.org/software/bash/manual/ bash manual].
| |
| | |
| If you want to see what commands you have run in the current login session you can run the command <code>history</code>. Bash also keeps history from previous sessions in the file <code>~/.bash_history</code>.
| |
| | |
| You should create the following directories and use (a) your <code>bin</code> directory to put programs or scripts that you have downloaded or created that you want to be able to use, (b) your <code>programs</code> directory to put installation files for programs that you might need, (c) your <code>private</code> directory to keep files you want to make sure to keep private from other users, and (d) your <code>public_html</code> directory to store files that you want to be accessible on the web (at https://cs.indstate.edu/~YOUR_LOGIN). | |
| <pre> | | <pre> |
| mkdir ~/bin/
| | git add some_file.txt # this "stages" the file to be part of the next commit |
| echo "export PATH=$PATH:~/bin/" >> .bashrc # add the ~/bin/ directory to where shell looks for programs
| | git commit -m "Some updates to the file" # This commits, will be a permanent record of this version of the file |
| mkdir ~/programs/
| | git push # sends all committed changes to the remote server |
| mkdir ~/private/
| |
| chmod go-rwx ~/private/ # makes that directory not accessible to other users
| |
| mkdir ~/public_html/
| |
| chmod go+rx ~/public_html/ # so the web server can access this directory
| |
| chmod go+x ~ # so the web server can access your public_html
| |
| </pre> | | </pre> |
| Note that the web server is Apache, so if you want to configure how it processes files in your account, you should look for help on the Apache web server. It is also possible to run a web server using node.js; you can look up tutorials for how to do that. | | Note that the push will likely ask for login information to the remote server. Note that if you want there to be a record of a particular version of a file, you need to stage/commit the file. It is the commit'ed versions of files that are stored permanently. |
| | |
| | '''Pull''' - To pull/download updates that have happened (by others), you use <code>git pull</code>. |
|
| |
|
| =Copying/Creating/Editing Files=
| | '''Merge''' - When you ready to have your changes merged in to the main branch of the project, ask the owner of the project to check your branch. When the owner is ready for it, the owner can run the following commands. |
| Create a file in your <code>public_html</code> directory, and practice copying a file from somewhere else on the system.
| |
| <pre>
| |
| cd ~/public_html
| |
| nano hello.txt # or use your preferred editor - vim, emacs, jove, ...
| |
| # type something in your hello.txt file, save it, and
| |
| # check that you can view it at https://cs.indstate.edu/~YOUR_LOGIN/hello.txt
| |
| cp /usr/dict/words . # copies that file into current directory, which should still be your public_html
| |
| # now check https://cs.indstate.edu/~YOUR_LOGIN/words
| |
| </pre>
| |
| Create a sub-directory within your <code>public_html</code> directory, put some file there, edit the Apache configuration file (<code>.htaccess</code>) to allow directory listing of this directory (turned off by default for security reasons).
| |
| <pre> | | <pre> |
| cd ~/public_html
| | git branch main # owner is working in the main branch |
| mkdir files_public
| | git merge branch_name # merge your branch into the main branch |
| cd files_public
| |
| echo "Options +Indexes" >> .htaccess
| |
| ls -a # verify that the .htaccess file exists now
| |
| cat .htaccess # see what is in the file now
| |
| cp /var/junk/kinne/text-samples/shakespeare.txt .
| |
| cp /var/junk/kinne/text-samples/austen.txt .
| |
| # Now browse to https://cs.indstate.edu/~YOUR_LOGIN/files_public | |
| </pre> | | </pre> |
|
| |
| =Long Running Jobs=
| |
| If you have a process that needs to run for more than a few minutes, you should run it like this:
| |
| <pre>time nice -n 19 whatever_command</pre>
| |
| Starting the command with <code>time</code> will give you a printout afterwards of how much time the job took (<code>man time</code> for more details). Having <code>nice -n 19</code> makes your long running job run at a lower priority. Your job will finish just as fast, but it will interfere less with more important processes (other users on the system).
| |
|
| |
| If the job is going to take an hour or more, you probably want to run it like this:
| |
| <pre>time nice -n 19 whatever_command > output.txt 2>stderr.txt &</pre>
| |
| The <code> > output.txt</code> will have output go to the output.txt file, and <code> 2>stderr.txt</code> (no spaces is important) redirects error output to stderr.txt. The <code>&</code> will detach the process from your terminal so that it can continue running even if your terminal disconnects. You can login again and check on it by running <code>ps a</code> to see all your running processes (not just the ones started on the current login).
| |
|
| |
| Alternatively to redirecting the output and detaching, you could use the <code>screen</code> command to run the command in a terminal that you can detach. You can lookup help on the screen command for how to do that (for example, [https://www.geeksforgeeks.org/screen-command-in-linux-with-examples/ here]).
| |
This page is part of Programming and CS - Getting Started
Git
Git is a version control system that can be used to keep track of your files (programming code, but also other files as well).
Two of the most popular installations of git (as of 2020) are github and gitlab. When Microsoft purchased github, it became fashionable to prefer gitlab. Github allows for projects to be hosted on their servers. Gitlab allows for hosting either on their servers or to run a local installation of gitlab on ones own servers.
Of late, we have hosted projects using gitlab either on gitlab's servers or on ISU's local installation of gitlab.
Account Setup
On gitlab.com - Account setup on gitlab.com is just like any other website. You create a free account, choose your username/password, etc. The account is yours and is not tied in any particular way to ISU. You can give access to your projects in much the same way as you give access to files on google drive, onedrive, dropbox, etc.
On gitlab.indstate.edu - ISU's local installation of gitlab is at https://gitlab.indstate.edu/. For this system you use your ISU portal id and password. You do not need to create an account. The first time you login into the system, it initializes an account for you. CS courses and ISU CS projects in general tend to use ISU's local installation because we can tie access based on course rosters, etc.
Getting Around
Whether you are using gitlab.com or gitlab.indstate.edu, the functionality and "how to get around" is about the same. You have two main methods for using either.
Web browser interface
You can view projects within your web browser. How to get around is fairly intuitive - as is the case with any web page, you look around and click around to see what different things do. When you have something specific you want to do but don't see how to do, you ask the internet or ask your instructor (and we may update this wiki to give instructions on how to do it).
Terminal interface
Git allows for downloading/uploading/etc. files in your projects from the terminal. In order to do this, you need to (a) have a git client installed on your system, (b) understand the terminology that git uses for keeping track of files and the steps in using git to download/update/upload files, (c) make it so.
Some reading material about using git - https://www.tutorialspoint.com/git/index.htm
Vocab / Cheat Sheet
Workflow
A common use case is that you will need to download someone else's (e.g., instructor's) gitlab project, create your own version of some part of the project, test/debug your version of the project, eventually put in a request to update the main project with your code. The steps involved in that workflow.
Clone - To download the current version of the project you (a) browse in your terminal to the directory where you want to have the project code, (b) run git clone url_to_project
Branch - If a branch has already been created for you, then you can set that as your active branch by running git checkout branch_name. If a branch has not yet been created for you, you run git branch branch_name to create a new branch. In either case, you have a branch of the project to work within.
Do your work - You work on your files locally - test, debug, etc.
Stage/Commit/Push - To save your work to the remote server, you need to stage, commit, and push. You can stage/commit batches of changes, or do a stage/commit for each discreet task accomplished. If you are working in some_file.txt, you would run the following sequence of commands.
git add some_file.txt # this "stages" the file to be part of the next commit
git commit -m "Some updates to the file" # This commits, will be a permanent record of this version of the file
git push # sends all committed changes to the remote server
Note that the push will likely ask for login information to the remote server. Note that if you want there to be a record of a particular version of a file, you need to stage/commit the file. It is the commit'ed versions of files that are stored permanently.
Pull - To pull/download updates that have happened (by others), you use git pull.
Merge - When you ready to have your changes merged in to the main branch of the project, ask the owner of the project to check your branch. When the owner is ready for it, the owner can run the following commands.
git branch main # owner is working in the main branch
git merge branch_name # merge your branch into the main branch