1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
|
GAWK(1) Utility Commands GAWK(1)
NAME
gawk - pattern scanning and processing language
SYNOPSIS
gawk [ POSIX or GNU style options ] -f program-file [ -- ] file ...
gawk [ POSIX or GNU style options ] [ -- ] program-text file ...
DESCRIPTION
Gawk is the GNU Project's implementation of the AWK programming language. It conforms to
the definition of the language in the POSIX 1003.2 Command Language And Utilities Stan-
dard. This version in turn is based on the description in The AWK Programming Language,
by Aho, Kernighan, and Weinberger, with the additional features defined in the System V
Release 4 version of UNIX awk. Gawk also provides some GNU-specific extensions.
The command line consists of options to gawk itself, the AWK program text (if not supplied
via the -f or --file options), and values to be made available in the ARGC and ARGV pre-
defined AWK variables.
OPTIONS
Gawk options may be either the traditional POSIX one letter options, or the GNU style long
options. POSIX style options start with a single ``-'', while GNU long options start with
``--''. GNU style long options are provided for both GNU-specific features and for POSIX
mandated features. Other implementations of the AWK language are likely to only accept
the traditional one letter options.
Following the POSIX standard, gawk-specific options are supplied via arguments to the -W
option. Multiple -W options may be supplied, or multiple arguments may be supplied to-
gether if they are separated by commas, or enclosed in quotes and separated by white
space. Case is ignored in arguments to the -W option. Each -W option has a corresponding
GNU style long option, as detailed below. Arguments to GNU style long options are either
joined with the option by an = sign, with no intervening spaces, or they may be provided
in the next command line argument.
Gawk accepts the following options.
-F fs
--field-separator=fs
Use fs for the input field separator (the value of the FS predefined variable).
-v var=val
--assign=var=val
Assign the value val, to the variable var, before execution of the program begins.
Such variable values are available to the BEGIN block of an AWK program.
-f program-file
--file=program-file
Read the AWK program source from the file program-file, instead of from the first
command line argument. Multiple -f (or --file) options may be used.
-mf=NNN
-mr=NNN
Set various memory limits to the value NNN. The f flag sets the maximum number of
fields, and the r flag sets the maximum record size. These two flags and the -m
option are from the AT&T Bell Labs research version of UNIX awk. They are ignored
by gawk, since gawk has no pre-defined limits.
-W compat
--compat Run in compatibility mode. In compatibility mode, gawk behaves identically to
UNIX awk; none of the GNU-specific extensions are recognized. See GNU EXTEN-
SIONS, below, for more information.
-W copyleft
-W copyright
--copyleft
--copyright Print the short version of the GNU copyright information message on the error
output.
-W help
-W usage
--help
--usage Print a relatively short summary of the available options on the error output.
Per the GNU Coding Standards, these options cause an immediate, successful
exit.
-W lint
--lint Provide warnings about constructs that are dubious or non-portable to other
AWK implementations.
-W posix
--posix This turns on compatibility mode, with the following additional restrictions:
+o \x escape sequences are not recognized.
+o The synonym func for the keyword function is not recognized.
+o The operators ** and **= cannot be used in place of ^ and ^=.
-W source=program-text
--source=program-text
Use program-text as AWK program source code. This option allows the easy in-
termixing of library functions (used via the -f and --file options) with
source code entered on the command line. It is intended primarily for medium
to large size AWK programs used in shell scripts.
The -W source= form of this option uses the rest of the command line argument
for program-text; no other options to -W will be recognized in the same argu-
ment.
-W version
--version Print version information for this particular copy of gawk on the error out-
put. This is useful mainly for knowing if the current copy of gawk on your
system is up to date with respect to whatever the Free Software Foundation is
distributing. Per the GNU Coding Standards, these options cause an immediate,
successful exit.
-- Signal the end of options. This is useful to allow further arguments to the
AWK program itself to start with a ``-''. This is mainly for consistency with
the argument parsing convention used by most other POSIX programs.
In compatibility mode, any other options are flagged as illegal, but are otherwise ig-
nored. In normal operation, as long as program text has been supplied, unknown options
are passed on to the AWK program in the ARGV array for processing. This is particularly
useful for running AWK programs via the ``#!'' executable interpreter mechanism.
AWK PROGRAM EXECUTION
An AWK program consists of a sequence of pattern-action statements and optional function
definitions.
pattern { action statements }
function name(parameter list) { statements }
Gawk first reads the program source from the program-file(s) if specified, from arguments
to -W source=, or from the first non-option argument on the command line. The -f and -W
source= options may be used multiple times on the command line. Gawk will read the pro-
gram text as if all the program-files and command line source texts had been concatenated
together. This is useful for building libraries of AWK functions, without having to in-
clude them in each new AWK program that uses them. It also provides the ability to mix
library functions with command line programs.
The environment variable AWKPATH specifies a search path to use when finding source files
named with the -f option. If this variable does not exist, the default path is
".:/usr/lib/awk:/usr/local/lib/awk". If a file name given to the -f option contains a
``/'' character, no path search is performed.
Gawk executes AWK programs in the following order. First, all variable assignments speci-
fied via the -v option are performed. Next, gawk compiles the program into an internal
form. Then, gawk executes the code in the BEGIN block(s) (if any), and then proceeds to
read each file named in the ARGV array. If there are no files named on the command line,
gawk reads the standard input.
If a filename on the command line has the form var=val it is treated as a variable assign-
ment. The variable var will be assigned the value val. (This happens after any BEGIN
block(s) have been run.) Command line variable assignment is most useful for dynamically
assigning values to the variables AWK uses to control how input is broken into fields and
records. It is also useful for controlling state if multiple passes are needed over a sin-
gle data file.
If the value of a particular element of ARGV is empty (""), gawk skips over it.
For each line in the input, gawk tests to see if it matches any pattern in the AWK pro-
gram. For each pattern that the line matches, the associated action is executed. The
patterns are tested in the order they occur in the program.
Finally, after all the input is exhausted, gawk executes the code in the END block(s) (if
any).
VARIABLES AND FIELDS
AWK variables are dynamic; they come into existence when they are first used. Their values
are either floating-point numbers or strings, or both, depending upon how they are used.
AWK also has one dimensional arrays; arrays with multiple dimensions may be simulated.
Several pre-defined variables are set as a program runs; these will be described as needed
and summarized below.
Fields
As each input line is read, gawk splits the line into fields, using the value of the FS
variable as the field separator. If FS is a single character, fields are separated by
that character. Otherwise, FS is expected to be a full regular expression. In the spe-
cial case that FS is a single blank, fields are separated by runs of blanks and/or tabs.
Note that the value of IGNORECASE (see below) will also affect how fields are split when
FS is a regular expression.
If the FIELDWIDTHS variable is set to a space separated list of numbers, each field is ex-
pected to have fixed width, and gawk will split up the record using the specified widths.
The value of FS is ignored. Assigning a new value to FS overrides the use of FIELDWIDTHS,
and restores the default behavior.
Each field in the input line may be referenced by its position, $1, $2, and so on. $0 is
the whole line. The value of a field may be assigned to as well. Fields need not be ref-
erenced by constants:
n = 5
print $n
prints the fifth field in the input line. The variable NF is set to the total number of
fields in the input line.
References to non-existent fields (i.e. fields after $NF) produce the null-string. How-
ever, assigning to a non-existent field (e.g., $(NF+2) = 5) will increase the value of NF,
create any intervening fields with the null string as their value, and cause the value of
$0 to be recomputed, with the fields being separated by the value of OFS. References to
negative numbered fields cause a fatal error.
Built-in Variables
AWK's built-in variables are:
ARGC The number of command line arguments (does not include options to gawk, or the
program source).
ARGIND The index in ARGV of the current file being processed.
ARGV Array of command line arguments. The array is indexed from 0 to ARGC - 1. Dy-
namically changing the contents of ARGV can control the files used for data.
CONVFMT The conversion format for numbers, "%.6g", by default.
ENVIRON An array containing the values of the current environment. The array is in-
dexed by the environment variables, each element being the value of that vari-
able (e.g., ENVIRON["HOME"] might be /u/arnold). Changing this array does not
affect the environment seen by programs which gawk spawns via redirection or
the system() function. (This may change in a future version of gawk.)
ERRNO If a system error occurs either doing a redirection for getline, during a read
for getline, or during a close(), then ERRNO will contain a string describing
the error.
FIELDWIDTHS A white-space separated list of fieldwidths. When set, gawk parses the input
into fields of fixed width, instead of using the value of the FS variable as
the field separator. The fixed field width facility is still experimental;
expect the semantics to change as gawk evolves over time.
FILENAME The name of the current input file. If no files are specified on the command
line, the value of FILENAME is ``-''. However, FILENAME is undefined inside
the BEGIN block.
FNR The input record number in the current input file.
FS The input field separator, a blank by default.
IGNORECASE Controls the case-sensitivity of all regular expression operations. If IGNORE-
CASE has a non-zero value, then pattern matching in rules, field splitting
with FS, regular expression matching with ~ and !~, and the gsub(), index(),
match(), split(), and sub() pre-defined functions will all ignore case when
doing regular expression operations. Thus, if IGNORECASE is not equal to
zero, /aB/ matches all of the strings "ab", "aB", "Ab", and "AB". As with all
AWK variables, the initial value of IGNORECASE is zero, so all regular expres-
sion operations are normally case-sensitive.
NF The number of fields in the current input record.
NR The total number of input records seen so far.
OFMT The output format for numbers, "%.6g", by default.
OFS The output field separator, a blank by default.
ORS The output record separator, by default a newline.
RS The input record separator, by default a newline. RS is exceptional in that
only the first character of its string value is used for separating records.
(This will probably change in a future release of gawk.) If RS is set to the
null string, then records are separated by blank lines. When RS is set to the
null string, then the newline character always acts as a field separator, in
addition to whatever value FS may have.
RSTART The index of the first character matched by match(); 0 if no match.
RLENGTH The length of the string matched by match(); -1 if no match.
SUBSEP The character used to separate multiple subscripts in array elements, by de-
fault "\034".
Arrays
Arrays are subscripted with an expression between square brackets ([ and ]). If the ex-
pression is an expression list (expr, expr ...) then the array subscript is a string con-
sisting of the concatenation of the (string) value of each expression, separated by the
value of the SUBSEP variable. This facility is used to simulate multiply dimensioned ar-
rays. For example:
i = "A" ; j = "B" ; k = "C"
x[i, j, k] = "hello, world\n"
assigns the string "hello, world\n" to the element of the array x which is indexed by the
string "A\034B\034C". All arrays in AWK are associative, i.e. indexed by string values.
The special operator in may be used in an if or while statement to see if an array has an
index consisting of a particular value.
if (val in array)
print array[val]
If the array has multiple subscripts, use (i, j) in array.
The in construct may also be used in a for loop to iterate over all the elements of an ar-
ray.
An element may be deleted from an array using the delete statement. The delete statement
may also be used to delete the entire contents of an array.
Variable Typing And Conversion
Variables and fields may be (floating point) numbers, or strings, or both. How the value
of a variable is interpreted depends upon its context. If used in a numeric expression, it
will be treated as a number, if used as a string it will be treated as a string.
To force a variable to be treated as a number, add 0 to it; to force it to be treated as a
string, concatenate it with the null string.
When a string must be converted to a number, the conversion is accomplished using atof(3).
A number is converted to a string by using the value of CONVFMT as a format string for
sprintf(3), with the numeric value of the variable as the argument. However, even though
all numbers in AWK are floating-point, integral values are always converted as integers.
Thus, given
CONVFMT = "%2.2f"
a = 12
b = a ""
the variable b has a string value of "12" and not "12.00".
Gawk performs comparisons as follows: If two variables are numeric, they are compared nu-
merically. If one value is numeric and the other has a string value that is a ``numeric
string,'' then comparisons are also done numerically. Otherwise, the numeric value is
converted to a string and a string comparison is performed. Two strings are compared, of
course, as strings. According to the POSIX standard, even if two strings are numeric
strings, a numeric comparison is performed. However, this is clearly incorrect, and gawk
does not do this.
Uninitialized variables have the numeric value 0 and the string value "" (the null, or
empty, string).
PATTERNS AND ACTIONS
AWK is a line oriented language. The pattern comes first, and then the action. Action
statements are enclosed in { and }. Either the pattern may be missing, or the action may
be missing, but, of course, not both. If the pattern is missing, the action will be exe-
cuted for every single line of input. A missing action is equivalent to
{ print }
which prints the entire line.
Comments begin with the ``#'' character, and continue until the end of the line. Blank
lines may be used to separate statements. Normally, a statement ends with a newline, how-
ever, this is not the case for lines ending in a ``,'', ``{'', ``?'', ``:'', ``&&'', or
``||''. Lines ending in do or else also have their statements automatically continued on
the following line. In other cases, a line can be continued by ending it with a ``\'', in
which case the newline will be ignored.
Multiple statements may be put on one line by separating them with a ``;''. This applies
to both the statements within the action part of a pattern-action pair (the usual case),
and to the pattern-action statements themselves.
Patterns
AWK patterns may be one of the following:
BEGIN
END
/regular expression/
relational expression
pattern && pattern
pattern || pattern
pattern ? pattern : pattern
(pattern)
! pattern
pattern1, pattern2
BEGIN and END are two special kinds of patterns which are not tested against the input.
The action parts of all BEGIN patterns are merged as if all the statements had been writ-
ten in a single BEGIN block. They are executed before any of the input is read. Similarly,
all the END blocks are merged, and executed when all the input is exhausted (or when an
exit statement is executed). BEGIN and END patterns cannot be combined with other pat-
terns in pattern expressions. BEGIN and END patterns cannot have missing action parts.
For /regular expression/ patterns, the associated statement is executed for each input
line that matches the regular expression. Regular expressions are the same as those in
egrep(1), and are summarized below.
A relational expression may use any of the operators defined below in the section on ac-
tions. These generally test whether certain fields match certain regular expressions.
The &&, ||, and ! operators are logical AND, logical OR, and logical NOT, respectively,
as in C. They do short-circuit evaluation, also as in C, and are used for combining more
primitive pattern expressions. As in most languages, parentheses may be used to change the
order of evaluation.
The ?: operator is like the same operator in C. If the first pattern is true then the pat-
tern used for testing is the second pattern, otherwise it is the third. Only one of the
second and third patterns is evaluated.
The pattern1, pattern2 form of an expression is called a range pattern. It matches all
input records starting with a line that matches pattern1, and continuing until a record
that matches pattern2, inclusive. It does not combine with any other sort of pattern ex-
pression.
Regular Expressions
Regular expressions are the extended kind found in egrep. They are composed of characters
as follows:
c matches the non-metacharacter c.
\c matches the literal character c.
. matches any character except newline.
^ matches the beginning of a line or a string.
$ matches the end of a line or a string.
[abc...] character class, matches any of the characters abc....
[^abc...] negated character class, matches any character except abc... and newline.
r1|r2 alternation: matches either r1 or r2.
r1r2 concatenation: matches r1, and then r2.
r+ matches one or more r's.
r* matches zero or more r's.
r? matches zero or one r's.
(r) grouping: matches r.
The escape sequences that are valid in string constants (see below) are also legal in reg-
ular expressions.
Actions
Action statements are enclosed in braces, { and }. Action statements consist of the usual
assignment, conditional, and looping statements found in most languages. The operators,
control statements, and input/output statements available are patterned after those in C.
Operators
The operators in AWK, in order of increasing precedence, are
= += -=
*= /= %= ^= Assignment. Both absolute assignment (var = value) and operator-assignment
(the other forms) are supported.
?: The C conditional expression. This has the form expr1 ? expr2 : expr3. If
expr1 is true, the value of the expression is expr2, otherwise it is expr3.
Only one of expr2 and expr3 is evaluated.
|| Logical OR.
&& Logical AND.
~ !~ Regular expression match, negated match. NOTE: Do not use a constant regular
expression (/foo/) on the left-hand side of a ~ or !~. Only use one on the
right-hand side. The expression /foo/ ~ exp has the same meaning as (($0 ~
/foo/) ~ exp). This is usually not what was intended.
< >
<= >=
!= == The regular relational operators.
blank String concatenation.
+ - Addition and subtraction.
* / % Multiplication, division, and modulus.
+ - ! Unary plus, unary minus, and logical negation.
^ Exponentiation (** may also be used, and **= for the assignment operator).
++ -- Increment and decrement, both prefix and postfix.
$ Field reference.
Control Statements
The control statements are as follows:
if (condition) statement [ else statement ]
while (condition) statement
do statement while (condition)
for (expr1; expr2; expr3) statement
for (var in array) statement
break
continue
delete array[index]
delete array
exit [ expression ]
{ statements }
I/O Statements
The input/output statements are as follows:
close(filename) Close file (or pipe, see below).
getline Set $0 from next input record; set NF, NR, FNR.
getline <file Set $0 from next record of file; set NF.
getline var Set var from next input record; set NF, FNR.
getline var <file Set var from next record of file.
next Stop processing the current input record. The next input record is
read and processing starts over with the first pattern in the AWK
program. If the end of the input data is reached, the END block(s),
if any, are executed.
next file Stop processing the current input file. The next input record read
comes from the next input file. FILENAME is updated, FNR is reset
to 1, and processing starts over with the first pattern in the AWK
program. If the end of the input data is reached, the END block(s),
if any, are executed.
print Prints the current record.
print expr-list Prints expressions. Each expression is separated by the value of
the OFS variable. The output record is terminated with the value of
the ORS variable.
print expr-list >file Prints expressions on file. Each expression is separated by the
value of the OFS variable. The output record is terminated with the
value of the ORS variable.
printf fmt, expr-list Format and print.
printf fmt, expr-list >file
Format and print on file.
system(cmd-line) Execute the command cmd-line, and return the exit status. (This may
not be available on non-POSIX systems.)
Other input/output redirections are also allowed. For print and printf, >>file appends
output to the file, while | command writes on a pipe. In a similar fashion, command |
getline pipes into getline. The getline command will return 0 on end of file, and -1 on
an error.
The printf Statement
The AWK versions of the printf statement and sprintf() function (see below) accept the
following conversion specification formats:
%c An ASCII character. If the argument used for %c is numeric, it is treated as a
character and printed. Otherwise, the argument is assumed to be a string, and the
only first character of that string is printed.
%d A decimal number (the integer part).
%i Just like %d.
%e A floating point number of the form [-]d.ddddddE[+-]dd.
%f A floating point number of the form [-]ddd.dddddd.
%g Use e or f conversion, whichever is shorter, with nonsignificant zeros suppressed.
%o An unsigned octal number (again, an integer).
%s A character string.
%x An unsigned hexadecimal number (an integer).
%X Like %x, but using ABCDEF instead of abcdef.
%% A single % character; no argument is converted.
There are optional, additional parameters that may lie between the % and the control let-
ter:
- The expression should be left-justified within its field.
width The field should be padded to this width. If the number has a leading zero, then
the field will be padded with zeros. Otherwise it is padded with blanks. This ap-
plies even to the non-numeric output formats.
.prec A number indicating the maximum width of strings or digits to the right of the dec-
imal point.
The dynamic width and prec capabilities of the ANSI C printf() routines are supported. A
* in place of either the width or prec specifications will cause their values to be taken
from the argument list to printf or sprintf().
Special File Names
When doing I/O redirection from either print or printf into a file, or via getline from a
file, gawk recognizes certain special filenames internally. These filenames allow access
to open file descriptors inherited from gawk's parent process (usually the shell). Other
special filenames provide access information about the running gawk process. The file-
names are:
/dev/pid Reading this file returns the process ID of the current process, in decimal,
terminated with a newline.
/dev/ppid Reading this file returns the parent process ID of the current process, in
decimal, terminated with a newline.
/dev/pgrpid Reading this file returns the process group ID of the current process, in dec-
imal, terminated with a newline.
/dev/user Reading this file returns a single record terminated with a newline. The
fields are separated with blanks. $1 is the value of the getuid(2) system
call, $2 is the value of the geteuid(2) system call, $3 is the value of the
getgid(2) system call, and $4 is the value of the getegid(2) system call. If
there are any additional fields, they are the group IDs returned by get-
groups(2). Multiple groups may not be supported on all systems.
/dev/stdin The standard input.
/dev/stdout The standard output.
/dev/stderr The standard error output.
/dev/fd/n The file associated with the open file descriptor n.
These are particularly useful for error messages. For example:
print "You blew it!" > "/dev/stderr"
whereas you would otherwise have to use
print "You blew it!" | "cat 1>&2"
These file names may also be used on the command line to name data files.
Numeric Functions
AWK has the following pre-defined arithmetic functions:
atan2(y, x) returns the arctangent of y/x in radians.
cos(expr) returns the cosine in radians.
exp(expr) the exponential function.
int(expr) truncates to integer.
log(expr) the natural logarithm function.
rand() returns a random number between 0 and 1.
sin(expr) returns the sine in radians.
sqrt(expr) the square root function.
srand(expr) use expr as a new seed for the random number generator. If no expr is pro-
vided, the time of day will be used. The return value is the previous seed
for the random number generator.
String Functions
AWK has the following pre-defined string functions:
gsub(r, s, t) for each substring matching the regular expression r in the string
t, substitute the string s, and return the number of substitu-
tions. If t is not supplied, use $0.
index(s, t) returns the index of the string t in the string s, or 0 if t is
not present.
length(s) returns the length of the string s, or the length of $0 if s is
not supplied.
match(s, r) returns the position in s where the regular expression r occurs,
or 0 if r is not present, and sets the values of RSTART and
RLENGTH.
split(s, a, r) splits the string s into the array a on the regular expression r,
and returns the number of fields. If r is omitted, FS is used in-
stead. The array a is cleared first.
sprintf(fmt, expr-list) prints expr-list according to fmt, and returns the resulting
string.
sub(r, s, t) just like gsub(), but only the first matching substring is re-
placed.
substr(s, i, n) returns the n-character substring of s starting at i. If n is
omitted, the rest of s is used.
tolower(str) returns a copy of the string str, with all the upper-case charac-
ters in str translated to their corresponding lower-case counter-
parts. Non-alphabetic characters are left unchanged.
toupper(str) returns a copy of the string str, with all the lower-case charac-
ters in str translated to their corresponding upper-case counter-
parts. Non-alphabetic characters are left unchanged.
Time Functions
Since one of the primary uses of AWK programs is processing log files that contain time
stamp information, gawk provides the following two functions for obtaining time stamps and
formatting them.
systime() returns the current time of day as the number of seconds since the Epoch (Mid-
night UTC, January 1, 1970 on POSIX systems).
strftime(format, timestamp)
formats timestamp according to the specification in format. The timestamp
should be of the same form as returned by systime(). If timestamp is missing,
the current time of day is used. See the specification for the strftime() func-
tion in ANSI C for the format conversions that are guaranteed to be available.
A public-domain version of strftime(3) and a man page for it are shipped with
gawk; if that version was used to build gawk, then all of the conversions de-
scribed in that man page are available to gawk.
String Constants
String constants in AWK are sequences of characters enclosed between double quotes (").
Within strings, certain escape sequences are recognized, as in C. These are:
\\ A literal backslash.
\a The ``alert'' character; usually the ASCII BEL character.
\b backspace.
\f form-feed.
\n new line.
\r carriage return.
\t horizontal tab.
\v vertical tab.
\xhex digits
The character represented by the string of hexadecimal digits following the \x. As
in ANSI C, all following hexadecimal digits are considered part of the escape se-
quence. (This feature should tell us something about language design by committee.)
E.g., "\x1B" is the ASCII ESC (escape) character.
\ddd The character represented by the 1-, 2-, or 3-digit sequence of octal digits. E.g.
"\033" is the ASCII ESC (escape) character.
\c The literal character c.
The escape sequences may also be used inside constant regular expressions (e.g.,
/[ \t\f\n\r\v]/ matches whitespace characters).
FUNCTIONS
Functions in AWK are defined as follows:
function name(parameter list) { statements }
Functions are executed when called from within the action parts of regular pattern-action
statements. Actual parameters supplied in the function call are used to instantiate the
formal parameters declared in the function. Arrays are passed by reference, other vari-
ables are passed by value.
Since functions were not originally part of the AWK language, the provision for local
variables is rather clumsy: They are declared as extra parameters in the parameter list.
The convention is to separate local variables from real parameters by extra spaces in the
parameter list. For example:
function f(p, q, a, b) { # a & b are local
..... }
/abc/ { ... ; f(1, 2) ; ... }
The left parenthesis in a function call is required to immediately follow the function
name, without any intervening white space. This is to avoid a syntactic ambiguity with
the concatenation operator. This restriction does not apply to the built-in functions
listed above.
Functions may call each other and may be recursive. Function parameters used as local
variables are initialized to the null string and the number zero upon function invocation.
The word func may be used in place of function.
EXAMPLES
Print and sort the login names of all users:
BEGIN { FS = ":" }
{ print $1 | "sort" }
Count lines in a file:
{ nlines++ }
END { print nlines }
Precede each line by its number in the file:
{ print FNR, $0 }
Concatenate and line number (a variation on a theme):
{ print NR, $0 }
SEE ALSO
egrep(1), getpid(2), getppid(2), getpgrp(2), getuid(2), geteuid(2), getgid(2), getegid(2),
getgroups(2)
The AWK Programming Language, Alfred V. Aho, Brian W. Kernighan, Peter J. Weinberger, Ad-
dison-Wesley, 1988. ISBN 0-201-07981-X.
The GAWK Manual, Edition 0.15, published by the Free Software Foundation, 1993.
POSIX COMPATIBILITY
A primary goal for gawk is compatibility with the POSIX standard, as well as with the lat-
est version of UNIX awk. To this end, gawk incorporates the following user visible fea-
tures which are not described in the AWK book, but are part of awk in System V Release 4,
and are in the POSIX standard.
The -v option for assigning variables before program execution starts is new. The book
indicates that command line variable assignment happens when awk would otherwise open the
argument as a file, which is after the BEGIN block is executed. However, in earlier im-
plementations, when such an assignment appeared before any file names, the assignment
would happen before the BEGIN block was run. Applications came to depend on this ``fea-
ture.'' When awk was changed to match its documentation, this option was added to accom-
modate applications that depended upon the old behavior. (This feature was agreed upon by
both the AT&T and GNU developers.)
The -W option for implementation specific features is from the POSIX standard.
When processing arguments, gawk uses the special option ``--'' to signal the end of argu-
ments. In compatibility mode, it will warn about, but otherwise ignore, undefined op-
tions. In normal operation, such arguments are passed on to the AWK program for it to
process.
The AWK book does not define the return value of srand(). The System V Release 4 version
of UNIX awk (and the POSIX standard) has it return the seed it was using, to allow keeping
track of random number sequences. Therefore srand() in gawk also returns its current seed.
Other new features are: The use of multiple -f options (from MKS awk); the ENVIRON array;
the \a, and \v escape sequences (done originally in gawk and fed back into AT&T's); the
tolower() and toupper() built-in functions (from AT&T); and the ANSI C conversion specifi-
cations in printf (done first in AT&T's version).
GNU EXTENSIONS
Gawk has some extensions to POSIX awk. They are described in this section. All the ex-
tensions described here can be disabled by invoking gawk with the -W compat option.
The following features of gawk are not available in POSIX awk.
+o The \x escape sequence.
+o The systime() and strftime() functions.
+o The special file names available for I/O redirection are not recognized.
+o The ARGIND and ERRNO variables are not special.
+o The IGNORECASE variable and its side-effects are not available.
+o The FIELDWIDTHS variable and fixed width field splitting.
+o No path search is performed for files named via the -f option. Therefore the
AWKPATH environment variable is not special.
+o The use of next file to abandon processing of the current input file.
+o The use of delete array to delete the entire contents of an array.
The AWK book does not define the return value of the close() function. Gawk's close() re-
turns the value from fclose(3), or pclose(3), when closing a file or pipe, respectively.
When gawk is invoked with the -W compat option, if the fs argument to the -F option is
``t'', then FS will be set to the tab character. Since this is a rather ugly special
case, it is not the default behavior. This behavior also does not occur if -W posix has
been specified.
HISTORICAL FEATURES
There are two features of historical AWK implementations that gawk supports. First, it is
possible to call the length() built-in function not only with no argument, but even with-
out parentheses! Thus,
a = length
is the same as either of
a = length()
a = length($0)
This feature is marked as ``deprecated'' in the POSIX standard, and gawk will issue a
warning about its use if -W lint is specified on the command line.
The other feature is the use of either the continue or the break statements outside the
body of a while, for, or do loop. Traditional AWK implementations have treated such usage
as equivalent to the next statement. Gawk will support this usage if -W compat has been
specified.
ENVIRONMENT VARIABLES
If POSIXLY_CORRECT exists in the environment, then gawk behaves exactly as if --posix had
been specified on the command line. If --lint has been specified, gawk will issue a warn-
ing message to this effect.
BUGS
The -F option is not necessary given the command line variable assignment feature; it re-
mains only for backwards compatibility.
If your system actually has support for /dev/fd and the associated /dev/stdin, /dev/std-
out, and /dev/stderr files, you may get different output from gawk than you would get on a
system without those files. When gawk interprets these files internally, it synchronizes
output to the standard output with output to /dev/stdout, while on a system with those
files, the output is actually to different open files. Caveat Emptor.
VERSION INFORMATION
This man page documents gawk, version 2.15.
Starting with the 2.15 version of gawk, the -c, -V, -C, -a, and -e options of the 2.11
version are no longer recognized. This fact will not even be documented in the manual
page for the next major version.
AUTHORS
The original version of UNIX awk was designed and implemented by Alfred Aho, Peter Wein-
berger, and Brian Kernighan of AT&T Bell Labs. Brian Kernighan continues to maintain and
enhance it.
Paul Rubin and Jay Fenlason, of the Free Software Foundation, wrote gawk, to be compatible
with the original version of awk distributed in Seventh Edition UNIX. John Woods contrib-
uted a number of bug fixes. David Trueman, with contributions from Arnold Robbins, made
gawk compatible with the new version of UNIX awk. Arnold Robbins is the current main-
tainer.
The initial DOS port was done by Conrad Kwok and Scott Garfinkle. Scott Deifik is the
current DOS maintainer. Pat Rankin did the port to VMS, and Michal Jaegermann did the
port to the Atari ST. The port to OS/2 was done by Kai Uwe Rommel, with contributions and
help from Darrel Hankerson.
BUG REPORTS
If you find a bug in gawk, please send electronic mail to bug-gnu-utils@prep.ai.mit.edu,
with a carbon copy to arnold@gnu.ai.mit.edu. Please include your operating system and its
revision, the version of gawk, what C compiler you used to compile it, and a test program
and data that are as small as possible for reproducing the problem.
Before sending a bug report, please do two things. First, verify that you have the latest
version of gawk. Many bugs (usually subtle ones) are fixed at each release, and if your's
is out of date, the problem may already have been solved. Second, please read this man
page and the reference manual carefully to be sure that what you think is a bug really is,
instead of just a quirk in the language.
ACKNOWLEDGEMENTS
Brian Kernighan of Bell Labs provided valuable assistance during testing and debugging.
We thank him.
Free Software Foundation Nov 24 1994 GAWK(1)
|