Showing posts with label command. Show all posts
Showing posts with label command. Show all posts

Tuesday, June 20, 2017

Posted by beni in , , , , , , , , | June 20, 2017

A Collection of 60 GNU sed Command Line Examples



This article is a long compilation of our six series of GNU sed command line examples. This article contains more than 60 command lines divided in 6 part. This is intended for those who want to practice, hence this article contains no explanation for every example. We hope this helps you to learn GNU sed easier.


GNU sed Examples The Series


  • Episode 1
  • Episode 2
  • Episode 3
  • Episode 4
  • Episode 5
  • Episode 6

Part 1


1. Read Text File

sed r text.txt


2. Find & Replace

sed �s/UNIX/MINIX/g� text.txt


3. Find & Replace (g Flag)

  1. echo �UNIX UNIX UNIX� | sed �s/UNIX/GNU/g�
  2. echo �UNIX UNIX UNIX� | sed �s/UNIX/GNU/�
4. Find & Replace (Uppercase)
  1. sed �s/have/U&/g� text.txt
5. Find & Replace (Lowercase)
  1. sed �s/UNIX/L&/g� text.txt
  2. sed �s/GNU/L&/g� text.txt
  3. echo �UNIX MINIX� | sed �s/MINIX/L&/g�
6. Delete All Lines
  1. sed �d� text.txt
  2. sed -i �d� text.txt
  3. sed �d� text.txt > new_text.txt
7. Delete Particular Line
  1. sed �1d� text.txt
  2. sed �1,3d� text.txt
  3. sed �4,6d� text.txt
8. Delete All Blank Lines
sed �/^$/d� text.txt

9. Reverse Delete Blank Lines
sed �/^$/!d� text.txt

10. Delete Lines Containing Particular String
sed �/UNIX/d� text.txt


Part 2


11. Find & Replace (Specific Line)
  1. sed 3 s/bsd/U&/g text2.txt
  2. sed 1 s/gnu/U&/g text2.txt
  3. sed 2 s/unix/U&/g text2.txt
12. Find & Replace (Column Number)
sed �s/gnu/U&/1� text2.txt

13. Find & Replace (Column Number & Specific Line)
  1. sed �1 s/gnu/U&/1� text2.txt
  2. sed �2 s/unix/U&/1� text2.txt
14. Find & Replace (Column Number & g Flag)
sed �1 s/gnu/U&/g1� text2.txt

15. Find & Replace (Duplicate Column)
  1. sed s/(bsd)/1 1 1/g text2.txt
  2. sed s/(gnu)/1 1 1/g text2.txt
16. Find & Replace (Decorate Column)
  1. sed s/(bsd)/[1]/g text2.txt
  2. sed s/(gnu)/{1}/g text2.txt
17. Find & Replace (Delete All Line Numbers)
sed s/^[0-9]*[0-9].//g text3.txt

18. Find & Replace (Case Insensitive)
sed s/bsd/changed/Ig text2.txt

19. Delete Line (Case Insensitive)
  1. sed /unix/Id text5.txt
  2. sed /bsd/Id text5.txt
  3. sed /gnu/Id text5.txt
20. Delete Line (OR Logic Regex)
  1. sed /<-|->/d text4.txt
  2. sed /sign|just/d text4.txt
  3. sed /sign|just|in/d text4.txt

Part 3

 

21. Run Multiple Commands
sed -e s/gnu/U&/g -e s/bsd/U&/g text6.txt

22. Run Multiple Commands (Simpler)
sed s/gnu/U&/g; s/bsd/U&/g text6.txt

23. Combine sed with bash Looping Commands
for i in {6..8}; do sed /bsd/d "text$i.txt"; echo ""; done;

24. Delete Contents of Multiple Files (bash Looping)
for i in {6..8}; do sed -i d text$i.txt; done

25. Rename Multiple Files (bash Looping)
  1. for i in *.txt; do mv "$i" "`echo $i | sed "s/text/flext/g"`"; done
  2. for i in *.txt; do mv -v "$i" "`echo $i | sed "s/^[0-9]*[0-9]//g"`"; done

26. Delete Multiple Files (bash Looping)
  1. for i in *.txt; do rm -v "`echo $i | sed "s/^[0-9]*[0-9]//g"`"; done
  2. for i in *.txt; do rm -v "`echo $i | sed -n "/^[0-9]*[0-9]/p"`"; done

27. Change Slash Delimiters to Another Characters
  1. sed s/unix/CHANGED/g text6.txt
  2. sed s@unix@CHANGED@g text6.txt
  3. sed s!unix!CHANGED!g text6.txt
  4. sed s?unix?CHANGED?g text6.txt

28. Adjust Line Spacing (Single)
sed �G� text6.txt

29. Adjust Line Spacing (Double)
sed G;G text6.txt

30. Find & Replace (Lines Range)
sed 1,3 s/unix/CHANGED/g text6.txt 


Part 4


31. Print Matched Lines
sed -n /unix/p text9.txt

32. Print Reverse-Matched Lines
sed -n /unix/!p text9.txt

33. Bash Looping with Delay Time
for i in *.txt; do sed s/x/[X]/g $i; echo " "; sleep 1; done

34. Print The File Name
sed -n $ F text9.txt

35. Print The First Line
sed �q� text9.txt

36. Print The Last Line
sed -n $ p text9.txt

37. Count Total Line Number
sed -n $ = text9.txt

38. Print Multiple Files First Lines (bash Looping)
  1. for i in *.txt; do sed �q� $i; done
  2. for i in *.txt; do sed -n $ F $i; sed q $i; done
39. Print Multiple Files Last Lines (bash Looping)
  1. for i in *.txt; do sed -n $ p $i; done
  2. for i in *.txt; do sed -n $ F $i; sed -n $ p $i; done
40. Count Multiple Files Line Numbers (bash Looping)
for i in *.txt; do sed -n $ F; $ = "$i"; done


Part 5


41. Delete Comment Lines (Double Slash Style)
sed /^//.*$/d text12.txt

42. Delete Comment Lines (Double Slash Style, Except At Line Beginning)
sed s+([^!]//.*$)++g text12.txt

43. Delete Comment Lines (Hash Style)
sed /^#[^!].*.$/d text13.txt

44. Delete Comment Lines (Slash-Asterisk Style)
sed //*/,/*//d text14.txt

45. Boolean Operator OR
sed -E -n /this|line|return/p text14.txt

46. Boolean Operator AND
  1. sed -n /this.*comment/p text14.txt
  2. sed -n /this.*slash.*comment.*/p text14.txt
47. Multiple Convert Images (PNG to JPEG)
for i in *.png; do convert -verbose "$i" "`echo $i | sed s/.png/.jpeg/g`"; done

48. Multiple Convert Images (JPEG to PNG)
for i in *.jpeg; do convert -verbose "$i" "`echo $i | sed s/.jpeg/.png/g`"; done

49. Insert A Single Line
  1. sed 1i # THIS IS A NEW LINE text13.txt
  2. sed 3i # THIS IS A NEW LINE text13.txt
50. Insert Multiple Lines
sed 3i # THIS IS A NEW LINE # THIS IS ANOTHER LINE text13.txt


Part 6


51. Print Only Lines Between Two Patterns
  1. sed -n /gnu/,/dunix/p text15.txt
  2. sed -n /dunix/,/sunos/Ip text15.txt
52. Delete Only Comment Lines Between Address Range
  1. sed 10,19{//*/,/*//d} text14.txt
  2. sed 10,19{////d} text14.txt
53. Delete Only Comment Lines Between Two Patterns
  1. sed /printf/,/printf/{//*/,/*//d} text14.txt
  2. sed /^/,/main/{//*/,/*//d} text14.txt
54. Edit Only Matched Lines Between Address Range
sed 4,7{s/x/[X]/Ig} text15.txt

55. Edit All Uppercase Characters (POSIX Character Class)
sed s/[[:upper:]]/[X]/g text15.txt

56. Edit All Lowercase Characters (POSIX Character Class)
sed s/[[:lower:]]/[X]/g text15.txt

57. Edit All Punctuation Characters (POSIX Character Class)
sed s/[[:punct:]]/[X]/g text15.txt

58. Edit All Uppercase & Lowercase (POSIX Character Class)
sed s/[[:alpha:]]/[X]/g text15.txt

59. Edit All Numeric Characters (POSIX Character Class)
sed s/[[:digit:]]/[X]/g text15.txt

60. Edit All Space & Tab Characters (POSIX Character Class)
sed s/[[:blank:]]/[X]/g text15.txt

Thursday, June 15, 2017

Posted by beni in , , , , , , , | June 15, 2017

A command to remove all Windows mapped drives


If you work in a Corp env and having a Laptop, you will likely go home with it still running and it must has all the network drives still mapped. And then later when you are at home and try to use your Laptop, you will see considerable delay. The Windows Explore and many Windows application that access File Browser dialog will try relentlessly in reconnecting those drives without sucess. This also affect your Cygwin shell too.

Here is a DOS command to remove them all.


C:> net use * /delete /y
If you are in Cygwin bash, then you must quote the *.

Sunday, May 28, 2017

Posted by beni in , , , , , , , | May 28, 2017

6 Easy ImageMagick Bulk Processing Command Line Examples



This is a short compilation of 6 bulk processing command line examples from 6 of our previous articles about bulk images converting, resizing, reducing (file size), rotating, and flipping. Here we mention only the title and the command lines, without the explanations (but we give a link to the corresponding article), for every action of processing below. For those dont know, the command lines here are basically just bash looping commands combined with ImageMagicks convert command lines.



1. Bulk Converting Images Format



PNG to JPEG:

for i in *.png; do convert -verbose "$i" "`echo $i | sed s/.png/.jpeg/g`"; done


JPEG to PNG:

for i in *.jpeg; do convert -verbose "$i" "`echo $i | sed s/.jpeg/.png/g`"; done

Corresponding article: http://www.ubuntubuzz.com/2016/07/bulk-converting-png-or-jpeg-with-imagemagick-command-lines.html.


2. Bulk Resizing (By Percentages)



25%:

for i in *.png; do convert -verbose -resize 25% "$i" "`echo $i | sed s/.png/-converted.png/g`"; done


50%:

for i in *.png; do convert -verbose -resize 50% "$i" "`echo $i | sed s/.png/-converted.png/g`"; done


75%:

for i in *.png; do convert -verbose -resize 75% "$i" "`echo $i | sed s/.png/-converted.png/g`"; done

Corresponding article http://www.ubuntubuzz.com/2016/07/bulk-resizing-images-with-imagemagick-command-lines.html.

3. Bulk Resizing (By Resolutions)



600x400:

for i in *.png; do convert -verbose -resize 600x400 "$i" "`echo $i | sed s/.png/-converted.png/g`"; done


800x600:

for i in *.png; do convert -verbose -resize 800x600 "$i" "`echo $i | sed s/.png/-converted.png/g`"; done


1024x768:

for i in *.png; do convert -verbose -resize 1024x768 "$i" "`echo $i | sed s/.png/-converted.png/g`"; done


Corresponding article: http://www.ubuntubuzz.com/2016/07/bulk-resizing-images-by-resolutions.html.

4. Bulk Reducing Files Size



PNG:

for i in *.png; do convert -verbose -colors 128 "$i" "`echo $i | sed s/.png/-converted.png/g`"; done



JPEG:
for i in *.jpeg; do convert -verbose -quality 50 "$i" "`echo $i | sed s/.jpeg/-converted.jpeg/g`"; done

Corresponding article: http://www.ubuntubuzz.com/2016/07/bulk-reducing-image-files-size-with-imagemagick-command-lines.html.

5. Bulk Rotating 



120 degrees

for i in *.png; do convert -verbose -rotate 120 "$i" "`echo $i | sed s/.png/-converted.png/g`"; done;


45 degrees

for i in *.png; do convert -verbose -rotate 45 "$i" "`echo $i | sed s/.png/-converted.png/g`"; done;


75 degrees

for i in *.png; do convert -verbose -rotate 75 "$i" "`echo $i | sed s/.png/-converted.png/g`"; done;


Corresponding article: http://www.ubuntubuzz.com/2016/07/bulk-rotating-image-files-with-imagemagick-command-lines.html.

6. Bulk Flipping 


Vertical: 

for i in *.png; do convert -verbose -flip "$i" "`echo $i | sed s/.png/-converted.png/g`"; done;


Horizontal: 

for i in *.png; do convert -verbose -flop "$i" "`echo $i | sed s/.png/-converted.png/g`"; done;


Corresponding article: http://www.ubuntubuzz.com/2016/07/bulk-flipping-image-files-with.html.

Monday, April 17, 2017

Posted by beni in , , , , , , | April 17, 2017

5 Command Prompt Tricks you dont know


CMD tricks

We tend to use the command line a lot now a days, so we decided to show you 5 tips that we use in the command prompt that you can not know, read on to know what they are.

Sends the output of a command to the clipboardNote: this works for any command.How many times you use the ipconfig command just copy and paste the result? You will never do it again because you can simply send the output directly to the clipboard.

     
ipconfig | ClipOpen the command prompt from a folderHave you ever opened a command line and cd commands entered endless trying to get into a folder? If the answer is yes, then you will be happy to know that you can save a lot of time to open a command prompt in a folder from Windows Explorer. All you have to do is hold down the Shift key while right-clicking on a folder and the option shown in the context menu.Command HistoryChances are that you have pressed the button until you reach the above commands, but this can be a problem when you are trying to locate a particular command. Another way to look beyond the order is to use the command DOSKEY.

     
doskey / historyDrag and drop the file to change the current routeAnother trick if youre not a fan of opening a command line from the command context menu is the ability to drag and drop folders and system will automatically enter the folder path. You must enter the CD command and drag the previous folder actually change the path, but you can use the same technique to a lot of different commands.Run multiple commands at onceOur final trick of the day is a command line that many geeks you know, the ability to run multiple commands at once, the connection with marriage and trade. You can do this with all controls and can connect as many as you want:

     
Ipconfig && netstat

Wednesday, April 12, 2017

Posted by beni in , , , , , | April 12, 2017

a few useful show IP command


sh ip route


R1#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route


Gateway of last resort is not set


     172.16.0.0/30 is subnetted, 2 subnets
C       172.16.1.0 is directly connected, Serial0/1
C       172.16.2.0 is directly connected, Serial0/0
     10.0.0.0/24 is subnetted, 1 subnets
D       10.1.1.0 [90/2195456] via 172.16.1.2, 00:02:01, Serial0/1
C    192.168.1.0/24 is directly connected, Loopback1

  • type of network ; static, connected
  • or how the router learn it ; EIGRP, OSPF.from which interface it learned

lets be more specific
show ip route [address]

R1#sh ip rou 10.1.1.1
Routing entry for 10.1.1.0/24
  Known via "eigrp 1", distance 90, metric 2195456, type internal
  Redistributing via eigrp 1
  Last update from 172.16.1.2 on Serial0/1, 00:07:16 ago
  Routing Descriptor Blocks:
  * 172.16.1.2, from 172.16.1.2, 00:07:16 ago, via Serial0/1
      Route metric is 2195456, traffic share count is 1
      Total delay is 21000 microseconds, minimum bandwidth is 1544 Kbit
      Reliability 255/255, minimum MTU 1500 bytes
      Loading 1/255, Hops 1

  • how its learned . ex; EIGRP, static or connected..
  • how long since the route learned, this can be useful when you want to check how long link down, up time.. etc,
  • the bandwidth information are reflected by the value configured on the interface .. what ever value you put, its gonna show the same in here

show ip route connected
show ip route static

  • issue this two cmd if you want to filter or be more specific on the type of route

show ip route summary

R1#sh ip rou sum
IP routing table name is Default-IP-Routing-Table(0)
IP routing table maximum-paths is 16
Route Source    Networks    Subnets     Overhead    Memory (bytes)
connected       1           2           216         408
static               0           0           0           0
eigrp 1            0           1           72          136
internal            2                                   2312
Total               3           3           288         2856
Removing Queue Size 0

  • this will summarize all you information base on Net, Subnet, Overhead(other memory used) and Memory


Sunday, April 2, 2017

Posted by beni in , , , , , , , , , , , , | April 02, 2017

A general approach to command line switches and their default values in Perl


In the UNIX world youll rarely find a program which doesnt support a few or many arguments (or command line parameters) which influence the execution of the program.

When Perl programs require arguments (one of the simplest cases: an input filename) one could investigate the ARGV hash (an approach which works well in easy cases) or one could turn to one of the Perl modules, in particular if the arguments are command line switches.

In this article I will discuss a few types of command line switches and the possible logic behind.

What is a command line switch?

Just to recap: a command line switch is traditionally denoted as a hyphen followed by a letter optionally followed by a value e.g.  -d or  -d 25 . Note the space between the switch and its value. Some programs require this space whereas other require the value to be attached to the switch like -d25 and still others allow both. Some programs allow switches to be concatenated like  -ltr instead of  -l -t -r . Others allow switches to be more than one letter. Some programs allow a switch to appear multiple times like -v in awk.
Further complexities exist: one switch might override others. Some switches exclude each other mutually.
All these cases would need to be handled properly.

On top of that (I think it was) the GNU world introduced double hyphen switches with (usually) string switches e.g.  --verbose .

In the remainder of this article I will only use the simple case of single letter switches with or without argument. I will be using Getopt::Std, one of the core Perl modules and its function getopts. Its basic usage is  getopts(ab:,%opts); for two switches  -a and  -b foo.

Various types of command line switches with or without default values

The typical distinction between command line switches is whether they are boolean (switched on or off) or carry an additional argument. Then there is the question: if a command line switch is absent should there be a default value used in the program?

The following table explains the differences and shows a few examples.

switchExampleDefaultNotes
 -a 
...falseA boolean switch by its very nature has a default value true or false which should be the opposite of what the switch intends to trigger.
 -d $HOME/tmp 
output directory/tmpCertain things in the program require a default value e.g. the program needs to know where to store its output files. Its left to the programmer to decide which of the default values can be overruled by command line switches.
 -u joe,sandy 
user listcurrent userSome command line switches can take more complex arguments, in this case a comma separated list of users. Its absence should be covered by a reasonable default value e.g. the current user.
 -p 1507 
process idall processesSome switches do specify a setting which acts as a filter or a kind of a restriction but its absence does not imply a default value but is somewhat vague.
In the user list example before another default behaviour could have been all users instead of current user.

Rather than defining a list of variables to set the defaults like

 $OUTDIR = "/tmp"; $USERS = $ENV{USER}; ... 
and later somehow associate these variables with the switches a (in my view) cleaner approach is to
  • define the defaults in a hash (the keys are the switches)
  • create a new hash (again with switches for keys) and set them to either the defaults or values supplied by the command line
    The following Perl program handles the cases above.
  • boolean switches and unspecified defaults are set to undef, all others are set to their reasonable default values.
  • the  ... ? ... : ... operator is used to set the actual variables
    (Getopt::Std sets boolean switches to 1 which represents true, the opposite (and default) could be anything that evaluates to false in an if(...) clause, I chose undef rather than 0).

     #!/usr/bin/perl use strict; use Getopt::Std; # to process command line arguments # Define the defaults in a hash my %defaults; $defaults{"a"} = undef; $defaults{"d"} = "/tmp"; $defaults{"u"} = $ENV{USER}; $defaults{"p"} = undef; # Retrieve the command line switches into a hash # making sure which ones are boolean and which require an argument with : my %opts; getopts(ad:u:p:,%opts); # Put either the default values or the command line switch arguments into a hash my %vars; foreach my $key (keys %defaults) { $vars{$key} = exists $opts{$key} ? $opts{$key} : $defaults{$key} ; } # Test output: see what is contained in vars foreach my $key (keys %vars) { print $key," ",$vars{$key}," "; } print " "; # Check decision tree for boolean and unspecified switches print "a is set " if( $vars{"a"} ); print "p: all processes " unless( $vars{"p"} ); 

    If run without any command line switches:

     u andreas p a d /tmp p: all processes 

    With -a and -u

     ... -a -u joe,sandy u joe,sandy p a 1 d /tmp a is set p: all processes 

    With -p and -d

     ... -d $HOME/tmp -p 1507 u andreas p 1507 a d /export/home/andreas/tmp 

    With this general approach one hash vars contains all the information and its contents can be used directly later in the program (like -d output directory) or used in a decision process defined vs. undefined.

    Of course there are more issues like the ones mentioned above (e.g. conflicting switches) or validity of values (e.g. does the output directory exist and is writable) but they need to be resolved somewhere else in the code.

  • Search