A simple implementation of UNIX |, <, <<, > and >> in pure C.

• About
• Checklist
• Getting Started
• Notes
• 42 Sao Paulo
• Resources

In this projects we implement UNIX pipes and redirections, namely |, <, <<, > and >>. These mechanisms allow us to link commands and files together, where the output of one serves as the input of the next.

This toolbox philosophy is one of the reasons why UNIX is still popular after 40 years: it encourages developers to create programs that do one thing well, and that work well with other programs. It's also extremely modular and versatile.

I learned about child processes, filesystem interfaces and the internals of bash, the target of our emulation. Since I did both bonuses I'm pretty confident about the next big challenge in the 42 curriculum: minishell. It's basically this with a parser and dynamic environment variables.

• DONT TURN IN LIBS AS SUBMODULES
• MAKEFILE EXPLICIT SOURCE FILES (echo M_SOURCES)
• Make must compile without relinking
  • SHOOULDNT RECOMPILE/REARCHIVE OBJECTS WITH MAKE ALL
    • Add .keep to object dirs
    • Remove initialize from all /build
    • Create non-phony rule for each lib archive
• .linux file (42 Workspaces)
• Test in workspaces
• Follows norminette 3.3.51
• Turn in Makefile, *.h, *.c , .linux , .gitignore
• Makefile rules: $(NAME) all clean fclean re
• Program name pipex
• Compiles with -Wall -Wextra -Werror
• Should not quit unexpectedly (segmentation fault, bus error, double free, etc.)
• All allocated heap memory properly freed, no memory leaks.
  • Check memory leaks with valgrind
• Allowed functions:
  • open, close, read, write, malloc, free, perror, strerror, access, dup, dup2, execve, exit, fork, pipe, unlink, wait, waitpid
  • libft allowed
  • Your ft_printf (may be modified)
    • No printf from stdio.h
• Handle arguments file1 cmd1 cmd2 file2
  • Wrong number of arguments exits with help message
• Behaves exactly like < file1 cmd1 | cmd2 > file2
  1. < file1 cmd1: < file1 read file1 and pipes it to cmd1's STDIN
  2. cmd1 | cmd2: cmd1's STDOUT is piped to cmd2's STDIN
  3. cmd2 > file2: cmd2's STDOUT is piped to file2, overwriting the file
• Handle errors exactly like < file1 cmd1 | cmd2 > file2
• Handle input errors (in order):
  • argc != 5 , return 0
  • No infile, return 0
  • No cmd1 , return 0
  • No cmd2 , return 127
  • cmd1 bad arguments, return 0
  • cmd2 bad arguments, return 1
• Pass all testers
  • https://github.com/denisgodoy/pipex-tester
  • https://github.com/gsilva-v/PipexTester
  • https://github.com/vfurmane/pipex-tester
  • https://github.com/Yoo0lh/pipex_tester_42

• Handle multiple pipes:
  • < file1 cmd1 | cmd2 | cmd3 ... | cmdn > file2
  • ./pipex file1 cmd1 cmd2 cmd3 ... cmdn file2
• Support << and >> when the first parameter is "here_doc":
  • ./pipex here_doc LIMITER cmd cmd1 file
  • cmd << LIMITER | cmd1 >> file

Clone the repo and build with make:

It works exactly like < infile cmd1 | cmd2 > outfile:

You can also compile the bonus implementation:

It handles multiple pipes < infile cmd1 | cmd2 | cmd3 | ... | cmdn > outfile:

It also takes a heredoc as input cmd1 << LIMITER | cmd2 >> file:

Redirect infile 's contents to cmd 's STDIN

Redirects previous cmd1's STDOUT to cmd2 's STDIN

Truncates outfile with cmd 's STDOUT

• https://www.man7.org/linux/man-pages/man3/perror.3.html

• https://www.man7.org/linux/man-pages/man3/errno.3.html

• https://linux.die.net/man/3/strerror
• https://www.geeksforgeeks.org/error-handling-c-programs/

• https://www.man7.org/linux/man-pages/man2/open.2.html
• https://jameshfisher.com/2017/02/24/what-is-mode_t/

• https://www.man7.org/linux/man-pages/man2/close.2.html

Checks if the calling process can access a specific file

• https://www.man7.org/linux/man-pages/man2/access.2.html

Adjust file descriptors so they reference the same file.

• https://www.man7.org/linux/man-pages/man2/dup.2.html
• https://www.geeksforgeeks.org/dup-dup2-linux-system-call/

Replaces the current process with a system call.

• https://jameshfisher.com/2017/02/05/how-do-i-use-execve-in-c/
• https://stackoverflow.com/questions/29615540/using-execve-in-c
• https://www.tutorialspoint.com/unix_system_calls/execve.htm
• https://www.man7.org/linux/man-pages/man2/execve.2.html
• https://stackoverflow.com/questions/5429141/what-happens-to-malloced-memory-after-exec-changes-the-program-image/5429592#5429592

Forks the parent process, creating a child process. The child process is almost identical to its parent, with a copy of its memory space at the time of fork.

• https://www.man7.org/linux/man-pages/man2/fork.2.html
• https://www.youtube.com/watch?v=cex9XrZCU14&list=PLfqABt5AS4FkW5mOn2Tn9ZZLLDwA3kZUY

Creates a pipe: a data channel through which processes can communicate. Emitting processes can write topipefd[1], and receiving ones can read from pipefd[0].

• https://www.man7.org/linux/man-pages/man2/pipe.2.html
• https://man7.org/linux/man-pages/man7/pipe.7.html
• https://www.programacaoprogressiva.net/2014/09/Pipes-em-C-Comunicao-entre-Processos-IPC-Interprocess-Communication.html

• https://linux.die.net/man/2/unlink

Wait for a process to change state.

• https://man7.org/linux/man-pages/man2/wait.2.html
• https://www.man7.org/linux/man-pages/man3/waitpid.3p.html

A string array with all the environment variables of the parent shell.

• https://www.geeksforgeeks.org/c-program-print-environment-variables/
• https://www.geeksforgeeks.org/command-line-arguments-in-c-cpp/
• https://stackoverflow.com/questions/54723587/what-does-envp-stand-for

• Threads are lighter: easier to create and destroy
• Processes don't share memory

A linked list in the control structure with all the allocated memory pointers. The interface function ft_lalloc allocates memory and adds the pointer to the list. The interface function ft_free_lalloc frees all pointers and the list.

Part of the larger 42 Network, 42 Sao Paulo is a software engineering school that offers a healthy alternative to traditional education:

• It doesn't have any teachers and classes.
• Students learn by cooperating and correcting each other's work (peer-to-peer learning).
• Its focus is as much on social skills as it is on technical skills.
• It's completely free to anyone that passes its selection process - The Piscine

It's an amazing school, and I'm grateful for the opportunity.

• https://wikiless.org/wiki/Pipeline_(Unix)?lang=en
• https://wikiless.org/wiki/Unix_philosophy?lang=en
• https://devhints.io/bash
• https://stackoverflow.com/questions/9834086/what-is-a-simple-explanation-for-how-pipes-work-in-bash
• https://stackoverflow.com/questions/21287848/function-to-find-the-maximum-number-of-pipes
• https://linuxconfig.org/introduction-to-named-pipes-on-bash-shell
• https://linuxhint.com/bash_pipe_tutorial/
• https://www.delftstack.com/howto/linux/pipes-in-bash/
• https://devdocs.io/bash/

• https://www.gnu.org/software/bash/manual/html_node/Pipelines.html
• https://www.gnu.org/software/bash/manual/html_node/Lists.html
• https://stackoverflow.com/questions/66511243/bash-pipe-execution-order
• https://unix.stackexchange.com/questions/37508/in-what-order-do-piped-commands-run
• https://linuxhint.com/bash_pipe_tutorial/

• https://www.howtoforge.com/linux-nm-command/
• https://www.tutorialspoint.com/unix_commands/nm.htm

• https://cs61.seas.harvard.edu/site/2021/
• https://cs61.seas.harvard.edu/site/2021/ProcessControl/
• https://cs61.seas.harvard.edu/site/2021/Section11/

• https://www.youtube.com/watch?v=6SA6S9Ca5-U

A more convenient version of execve.

• https://linux.die.net/man/3/execlp

• https://stackoverflow.com/questions/16847385/valgrind-to-detect-memory-leak-in-child-process

• https://github.com/denisgodoy/pipex-tester
• https://github.com/vfurmane/pipex-tester
• https://github.com/Yoo0lh/pipex_tester_42
• https://github.com/HEADLIGHTER/pipexfasttest
• https://github.com/gmarcha/pipexMedic
• https://github.com/gsilva-v/PipexTester
• https://github.com/mariadaan/PIPEXaminator

• https://towardsdatascience.com/why-are-there-so-many-tokenization-methods-for-transformers-a340e493b3a8
• https://nlp.stanford.edu/IR-book/html/htmledition/tokenization-1.html

• https://stackoverflow.com/questions/2407605/c-warning-missing-sentinel-in-function-call

• https://stackoverflow.com/questions/36584062/should-i-free-memory-before-exit
• https://newbedev.com/should-i-free-memory-before-exit
• https://cboard.cprogramming.com/c-programming/75239-freeing-memory-before-exit.html
• http://c-faq.com/malloc/freeb4exit.html
• https://www.linuxquestions.org/questions/programming-9/to-free-or-not-to-free-before-an-exit-458107/
• https://stackoverflow.com/questions/654754/what-really-happens-when-you-dont-free-after-malloc-before-program-termination
• https://stackoverflow.com/questions/2975831/is-leaked-memory-freed-up-when-the-program-exits

• git clone --recurse-submodule REMOTE_REPO
• git submodule add REMOTE_REPO
• git submodule foreach git pull
• git submodule update --init --recursive
• https://stackoverflow.com/questions/33714063/how-to-update-submodules-in-git
• https://stackoverflow.com/questions/59271919/how-to-clone-public-submodule-in-github-actions
• https://stackoverflow.com/questions/50254184/git-submodule-and-fetch
• https://www.w3docs.com/snippets/git/how-to-add-a-submodule-in-git.html
• https://stackoverflow.com/questions/1260748/how-do-i-remove-a-submodule#1260982
• https://stackoverflow.com/questions/2006172/git-how-to-reset-a-remote-git-repository-to-remove-all-commits

• https://linuxize.com/post/bash-shebang/
• https://stackoverflow.com/questions/18219262/evaluating-variables-in-a-string
• https://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux
• https://stackoverflow.com/questions/1951742/how-can-i-symlink-a-file-in-linux
• https://stackoverflow.com/questions/9452935/unix-create-path-of-folders-and-file

• https://www.youtube.com/watch?v=6xbLgZpOBi8
• https://www.youtube.com/playlist?list=PLFAC320731F539902
• https://www.youtube.com/results?search_query=c+pipe+fork+wait
• https://www.youtube.com/playlist?list=PLfqABt5AS4FkW5mOn2Tn9ZZLLDwA3kZUY