Marzo 31

How to install and integrate OpenDKIM with Postfix

What is OpenDKIM?
It is a digital email signing/verification technology, which is already supported by some common mail providers. In general, DKIM means digitally signing all messages on the mail-server to verify the message was actually sent from the domain in question and was not spam

UPDATE THE SYSTEM

Before going any further, make sure you’re in a screen session and your system is fully up-to-date by running:

## screen -U -S opendkim-screen
## yum update

ENABLE EPEL REPOSITORY

OpenDKIM is available in the EPEL repository, so we need to enable it on the system before we can install OpenDKIM

## wget -P /tmp http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm
## rpm -Uvh /tmp/epel-release-6-8.noarch.rpm
## rm -f /tmp/epel-release-6-8.noarch.rpm

INSTALL OPENDKIM

Install the package using yum:

## yum install opendkim

CONFIGURE OPENDKIM

Next thing to do is to configure OpenDKIM. Its main configuration file is located in /etc/opendkim.conf, so before making any changes create a backup and add/edit the following:

## cp /etc/opendkim.conf{,.orig}
## vim /etc/opendkim.conf
AutoRestart             Yes
AutoRestartRate         10/1h
LogWhy                  Yes
Syslog                  Yes
SyslogSuccess           Yes
Mode                    sv
Canonicalization        relaxed/simple
ExternalIgnoreList      refile:/etc/opendkim/TrustedHosts
InternalHosts           refile:/etc/opendkim/TrustedHosts
KeyTable                refile:/etc/opendkim/KeyTable
SigningTable            refile:/etc/opendkim/SigningTable
SignatureAlgorithm      rsa-sha256
Socket                  inet:8891@localhost
PidFile                 /var/run/opendkim/opendkim.pid
UMask                   022
UserID                  opendkim:opendkim
TemporaryDirectory      /var/tmp

SET-UP PUBLIC/PRIVATE KEYS

Generate set of keys for your mydomain.com domain name:

## mkdir /etc/opendkim/keys/mydomain.com
## opendkim-genkey -D /etc/opendkim/keys/mydomain.com/ -d mydomain.com -s default
## chown -R opendkim: /etc/opendkim/keys/mydomain.com
## mv /etc/opendkim/keys/mydomain.com/default.private /etc/opendkim/keys/mydomain.com/default

add mydomain.com to OpenDKIM’s key table by adding the following record in /etc/opendkim/KeyTable

default._domainkey.mydomain.com mydomain.com:default:/etc/opendkim/keys/mydomain.com/default

next, edit /etc/opendkim/SigningTable and add the following record to OpenDKIM’s signing table:

*@mydomain.com default._domainkey.mydomain.com

and add your domain and your hostname as trusted hosts in /etc/opendkim/TrustedHosts:

127.0.0.1
mydomain.com
host.mydomain.com

assuming the domain in question is ‘mydomain.com’ and server’s hostname is set to ‘host.mydomain.com’

finally, edit your mydomain.com DNS zone and add the TXT record from/etc/opendkim/keys/mydomain.com/default.txt

default._domainkey      IN      TXT     ( "v=DKIM1; k=rsa; "
          "p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDApHRr7ZmXRaAB+RQRbP4VdMwIrIHIP18KFtXRsv/xpWc0Gix6ZXN13fcG03KNGKZo2PY+csPkGC5quDnH5V0JEhDZ78KcDWFsU6u4fr9ktVAdt6P7jWXjcyqdHOZ8+YN4cAeU4lRFNgQvdupIcByYwzPYMgBFHfJm9014HvRqhwIDAQAB" )  ; ----- DKIM key default for mydomain.com

it is also a good idea to add an SPF record if you haven’t already

mydomain.com. 14400 IN TXT "v=spf1 a mx ~all"

you can verify your dkim TXT record is valid using dig for example:

## dig +short default._domainkey.mydomain.com TXT

"v=DKIM1\; k=rsa\; " "p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDApHRr7ZmXRaAB+RQRbP4VdMwIrIHIP18KFtXRsv/xpWc0Gix6ZXN13fcG03KNGKZo2PY+csPkGC5quDnH5V0JEhDZ78KcDWFsU6u4fr9ktVAdt6P7jWXjcyqdHOZ8+YN4cAeU4lRFNgQvdupIcByYwzPYMgBFHfJm9014HvRqhwIDAQAB"

CONFIGURE POSTFIX

In order to integrate OpenDKIM with Postfix we need to add the following few lines in/etc/postfix/main.cf:

smtpd_milters           = inet:127.0.0.1:8891
non_smtpd_milters       = $smtpd_milters
milter_default_action   = accept
milter_protocol         = 2

(RE)START SERVICES

Add OpenDKIM to your system’s start-up and start opendkim and restart postfix using the following commands:

## service opendkim start
## chkconfig opendkim on
## service postfix restart

TEST THE SET-UP

To test the set-up simply send an email to  autorespond+dkim@dk.elandsys.com and you should receive back an email containing something like this:

==========================================================
Summary of Results
==========================================================
SPF check:          pass
DomainKeys check:   neutral
DKIM check:         pass
DKIM check:         pass
Category: POSTFIX | Los comentarios están deshabilitados en How to install and integrate OpenDKIM with Postfix
Marzo 21

DNS Amplification DDoS Attack – ISC BIND

This example works for anyone running ISC BIND

dns attack isc.org any query

I normally do not work with windows too much, but being on call this week I ended up having to fix a problem on a Windows 2008 server. I didn’t find any documentation online, so I figured I’d add this post.

For anyone running Parallels Plesk (unknown version, but I know our web admin always keeps these up to date) make sure you lock down your ISC BIND instance. If not, you will probably run into a DNS amplification attack which will cause named.exe to used ALL your memory and probably even crash.

2013-04-22 Update: Plesk was set to only allow localnets recursion, however the built in localnets acl seems to be broken.

"localnets" - matches all the IP address(es) and subnetmasks of the server on which BIND is running. For example, if the server has a single interface with an IP address of 192.168.2.3 and a netmask of 255.255.255.0 (or 192.168.2.2/24) then localnets will match 192.168.2.0 to 192.168.2.255 and 127.0.0.1 (the loopback is always present and has a single address, that is a netmask of 255.255.255.255). Some systems do not provide a way to determine the prefix lengths of local IPv6 addresses. In such a case, localnets only matches the local IP addresses, just like localhost though in this case it will apply to external and internal (same host) requests.

Are you affected?

tcpdump:

1
<code>12:28:00.121351 IP x.x.x.x.19135 &gt; x.x.x.x.53: 10809+ [1au] ANY? isc.org. (36)</code>

bind logs:

1
2
3
4
5
<code>12:28:00.643 client x.x.x.x#49046: query: isc.org IN ANY +ED (x.x.x.x)
12:28:00.644 client x.x.x.x#25135: query: isc.org IN ANY +ED (x.x.x.x)
12:28:00.645 client x.x.x.x#19771: query: isc.org IN ANY +ED (x.x.x.x)
12:28:00.646 client x.x.x.x#44031: query: isc.org IN ANY +ED (x.x.x.x)
12:28:00.647 client x.x.x.x#31518: query: isc.org IN ANY +ED (x.x.x.x)</code>

test with dig:

NOTE: x.x.x.x is YOUR dns server ip
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
<code>$ dig ANY isc.org @x.x.x.x +edns=0

; &lt;&lt;&gt;&gt; DiG 9.8.1-P1 &lt;&lt;&gt;&gt; ANY isc.org @x.x.x.x +edns=0
;; global options: +cmd
;; Got answer:
;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NOERROR, id: 8809
;; flags: qr rd ra; QUERY: 1, ANSWER: 30, AUTHORITY: 4, ADDITIONAL: 13

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;isc.org.                       IN      ANY

;; ANSWER SECTION:
isc.org.                300     IN      RRSIG   SPF 5 2 7200 20130515233253 20130415233253 50012 isc.org. rxkc3Rv5XIuARc+jiKh2DQIc3osweF+a7Db1OA8bbaEPfVW6eje0CrWM 1+D9gU0ghbNp4On4G4jfClcWTpefqhcGIzODMIwKXlGDcp1Y4t7f1Xt8 sAt0iDj7k4qoUOXEtcLoo3fhi0/HfjmJojNmNfzRuIy0q4VBPLvXDzY1 wio=
isc.org.                300     IN      SPF     "v=spf1 a mx ip4:204.152.184.0/21 ip4:149.20.0.0/16 ip6:2001:04F8::0/32 ip6:2001:500:60::65/128 ~all"
isc.org.                300     IN      RRSIG   DNSKEY 5 2 7200 20130515230132 20130415230132 12892 isc.org. By1JLKWH4p/NijvP7TO40IkAokI2o5w2tZlw1d92Iv7chSKkhnBlS0jh Hpo5IySLsr3yYmKnb5rv/lIMhlPVF5TUC3+ToY7hz6aouS5P4JYA1bIB SZlzxS5HAAPl3UddF4cwf5Dp3JON3E6VIzA588PMjUBD666A27JRNqut EbHI2WxnZBR9inxwDnEf5JPagEYgNMlADottLSa3PKxwtmWUS3OLZaOo 4+wMgbL+bqTI5h5y6IpOipz3rUWurFbYteTIy5RjC+uaLcazEM94G41Z YoXQP+LodcZTqiYnfbT0Cp3ahr3n+Kx3OHLglW/V5GoqyTDFjRrHtObc j1dA5A==
isc.org.                300     IN      RRSIG   DNSKEY 5 2 7200 20130515230132 20130415230132 50012 isc.org. GRdD8E7BJR+sD7V0MkBzWKeonk97axGU7sinBrc6szaons7LY1TWyn3T 5cllz850C6o5dlK22zykRjrwCI6wiuJVzuJbyAOjrwM3TtEjFv7ePAaK ad3VGofZeb0klGEtvG9L/5rMkF2bbAwxeFGuD7SPz1gsGKDurmbCQ8YD diw=
isc.org.                300     IN      DNSKEY  257 3 5 BEAAAAOhHQDBrhQbtphgq2wQUpEQ5t4DtUHxoMVFu2hWLDMvoOMRXjGr hhCeFvAZih7yJHf8ZGfW6hd38hXG/xylYCO6Krpbdojwx8YMXLA5/kA+ u50WIL8ZR1R6KTbsYVMf/Qx5RiNbPClw+vT+U8eXEJmO20jIS1ULgqy3 47cBB1zMnnz/4LJpA0da9CbKj3A254T515sNIMcwsB8/2+2E63/zZrQz Bkj0BrN/9Bexjpiks3jRhZatEsXn3dTy47R09Uix5WcJt+xzqZ7+ysyL KOOedS39Z7SDmsn2eA0FKtQpwA6LXeG2w+jxmw3oA8lVUgEf/rzeC/bB yBNsO70aEFTd
isc.org.                300     IN      DNSKEY  256 3 5 BQEAAAABwuHz9Cem0BJ0JQTO7C/a3McR6hMaufljs1dfG/inaJpYv7vH XTrAOm/MeKp+/x6eT4QLru0KoZkvZJnqTI8JyaFTw2OM/ItBfh/hL2lm Cft2O7n3MfeqYtvjPnY7dWghYW4sVfH7VVEGm958o9nfi79532Qeklxh x8pXWdeAaRU=
isc.org.                300     IN      RRSIG   NSEC 5 2 3600 20130515233253 20130415233253 50012 isc.org. oBP67HE1s2DNfMZ7z/MR6aY2Ujhq7PNY4eMuw6Y+i+qXbUbQ48T2O6Kj ATc9heRYWryRFQNj/JXvhDZvB4/4WsdgGqy7GdbNtBM5tSTKyp+/omc2 Bzg2y70TQgwoXyMnqCTUh/L5OAvLO8kk0xY1mbDSiv7l+gSAqQws876R zmI=
isc.org.                300     IN      NSEC    _adsp._domainkey.isc.org. A NS SOA MX TXT AAAA NAPTR RRSIG NSEC DNSKEY SPF
isc.org.                300     IN      RRSIG   NAPTR 5 2 7200 20130515233253 20130415233253 50012 isc.org. k+sm/1z+7Tp+cYeZL/IZHfGT4gVXG3Lto7n1bxCU34hh1DtuKYWXYra4 UdSpZ8lrFu+y4BMTVtR9eoDI2azQCbwJwU9E+btAsf7ZYRYONY7YkttZ l3iqckDHfNPb80/o25QRDX1VejYq9oSfOiKRVNjCvR9YvUMptPZJEn06 iYQ=
isc.org.                300     IN      NAPTR   20 0 "S" "SIP+D2U" "" _sip._udp.isc.org.
isc.org.                300     IN      RRSIG   AAAA 5 2 7200 20130515233253 20130415233253 50012 isc.org. M3h/PV6Fq3U0g45Z3FiwG+xcsnAok2T/nJwis4x/5MCuGk1wPRj1uUI+ h8nPPXkG9fq40uWV8hd7zKvZ5mSO0sDgdFDdkZLyPtG4jv67nw7/vUwb Pm3cqxTcMCSdKpPQvFXhq4X+bWdYJoNwNHGtviimMse8fdUER6dZqhmB Cqc=
isc.org.                300     IN      AAAA    2001:4f8:0:2::d
isc.org.                300     IN      RRSIG   TXT 5 2 7200 20130515233253 20130415233253 50012 isc.org. bdioDXnXaXz6+DRzq0WSEihIBcs7/5xbS7YzcEeCNdeq7c+dpD3SBNxj mG40ic0YH0ADYzXSi+MrgESZoQZmzUmBvw84FJcxtv1OHP1RYun9Scny acpx0U6SGhqvNFs4UaiJmPDYyJFaJcbGk47bQgTfbKDzDyCjhmbuEaVf 1fM=
isc.org.                300     IN      TXT     "v=spf1 a mx ip4:204.152.184.0/21 ip4:149.20.0.0/16 ip6:2001:04F8::0/32 ip6:2001:500:60::65/128 ~all"
isc.org.                300     IN      TXT     "$Id: isc.org,v 1.1793 2013-04-09 00:33:46 bind Exp $"
isc.org.                300     IN      RRSIG   MX 5 2 7200 20130515233253 20130415233253 50012 isc.org. HdmWOHdVRLEiKIgkeGAueVl5IhUVL2b8VddQwUAatNGIidhPHkSuylE4 E8+uh2qN/sTeYPI6DN7exvKvz+4i1dsO8lUXJqY2JY8JRizRb6oBPhgP wySTiM6mAfOzB6LFMy6bqb3IYiOApuToT/785I2WxzNPK0vmxGh29nH7 Tsc=
isc.org.                300     IN      MX      10 mx.pao1.isc.org.
isc.org.                300     IN      MX      10 mx.ams1.isc.org.
isc.org.                300     IN      RRSIG   SOA 5 2 7200 20130515233253 20130415233253 50012 isc.org. Qw/kiSPp8fgeYDucYo/tJP+SuQEWMoXJM9XPAgvuUj4ek44hUxjNUo2p 1VpLXaTIUnv6WdBvgo7X8wdPJckRag56y2zJSVbrAS3PrIALj9j90pbB 55RRc0xaGNusBU27orI4acFEZCpIH8yNv7zLx1MX3cWohoCisWaQMBwB Syo=
isc.org.                300     IN      SOA     ns-int.isc.org. hostmaster.isc.org. 2013041600 7200 3600 24796800 3600
isc.org.                300     IN      RRSIG   NS 5 2 7200 20130515233253 20130415233253 50012 isc.org. HUXmb89gB4pVehWRcuSkJg020gw2d8QMhTrcu1ZD7nKomXHQFupXl5vT iq5VUREGBQtnT7FEdPEJlCiJeogbAmqt3F1V5kBfdxZLe/EzYZgvSGWq sy/VHI5d+t6/EiuCjM01UXCH1+L0YAqiHox5gsWMzRW2kvjZXhRHE2+U i1Q=
isc.org.                300     IN      RRSIG   A 5 2 7200 20130515233253 20130415233253 50012 isc.org. LU6oTkxGAxOlX6zGFzwW+OSOXDZOi0NbPQGnF/evVZ+4N5FUN+7uApXP cTiwiXnX41JOqCpQyb/5zkUNyrl99/g5quVeOan06gBzsJfK8EoOSrOK UwbOKoJC1uWisJvIVS9+sb729LJ3bnAOFw0SDc22KLSaFKiwuKeTCtV6 Tuk=
isc.org.                300     IN      A       149.20.64.42
isc.org.                297     IN      RRSIG   DS 7 2 86400 20130508155535 20130417145535 31380 org. Fnk6ZJwwnJXTyIfgpFsax5dOeLFsEVg08GVeAgAaKiX/h2Oo4GZ/iVWu X9EAVCiL+hRCcpVsKED3Le+hzV4nazBtITsxTso3UTtWdJ49jJSRfsld 9Tp+5/Slf5dzwU4ST0d5VO4V5XN7mxTbIuAn9hpQ5ujVdZHjQYw/FkAU Bek=
isc.org.                297     IN      DS      12892 5 2 F1E184C0E1D615D20EB3C223ACED3B03C773DD952D5F0EB5C777586D E18DA6B5
isc.org.                297     IN      DS      12892 5 1 982113D08B4C6A1D9F6AEE1E2237AEF69F3F9759
isc.org.                297     IN      NS      ord.sns-pb.isc.org.
isc.org.                297     IN      NS      sfba.sns-pb.isc.org.
isc.org.                297     IN      NS      ams.sns-pb.isc.org.
isc.org.                297     IN      NS      ns.isc.afilias-nst.info.

;; AUTHORITY SECTION:
isc.org.                297     IN      NS      ams.sns-pb.isc.org.
isc.org.                297     IN      NS      ord.sns-pb.isc.org.
isc.org.                297     IN      NS      sfba.sns-pb.isc.org.
isc.org.                297     IN      NS      ns.isc.afilias-nst.info.

;; ADDITIONAL SECTION:
mx.ams1.isc.org.        300     IN      A       199.6.1.65
mx.ams1.isc.org.        300     IN      AAAA    2001:500:60::65
mx.pao1.isc.org.        300     IN      A       149.20.64.53
mx.pao1.isc.org.        300     IN      AAAA    2001:4f8:0:2::2b
ns.isc.afilias-nst.info. 299    IN      A       199.254.63.254
ns.isc.afilias-nst.info. 299    IN      AAAA    2001:500:2c::254
ams.sns-pb.isc.org.     297     IN      A       199.6.1.30
ams.sns-pb.isc.org.     297     IN      AAAA    2001:500:60::30
ord.sns-pb.isc.org.     297     IN      A       199.6.0.30
ord.sns-pb.isc.org.     297     IN      AAAA    2001:500:71::30
sfba.sns-pb.isc.org.    297     IN      A       149.20.64.3
sfba.sns-pb.isc.org.    297     IN      AAAA    2001:4f8:0:2::19

;; Query time: 4300 msec
;; SERVER: x.x.x.x#53(x.x.x.x)
;; WHEN: Sat Apr 20 18:20:04 2013
;; MSG SIZE  rcvd: 3628
</code>

To fix this and lock down BIND edit: C:\Parallels\Plesk\dns\etc\named.user.conf

Disable recursion for the DNS service
1
2
3
<code>options {
    recursion no;
};
</code>
ONLY Allow queries from known networks
1
2
3
<code>options {
    allow-query {192.168.1.0/24;};
};
</code>
  • Optionally, you can also use ACL’s for a cleaner/easier config
notice we also allow the trusted acl recursion
1
2
3
4
5
6
7
8
9
10
11
<code>acl trusted {
        127.0.0.1;
        127.0.0.0/24;
        172.16.0.0/16;
        192.168.1.0/24;
}

options {
    allow-recursion    {trusted; };
    allow-query        {trusted; };
};
</code>

2013-04-22 Update: Plesk will overwrite “allow-recursion { trusted; };” with the selection you have set in the web gui dns configuration with any, localnets or none. If you choose localnets (good choice), this feature may not work properly, so you will have to set the named.user.conf file immutable (read-only). If you do not, your changes *will revert.

2013-04-22 Update #2: *DO NOT set your file read only. Plesk Panel will break.. apperently it needs read/write access just to start. How about we just disable recursion. No need for this to be used as a resursive DNS server. It only needs to serve zones it’s authority for. You can disable recursion as a whole in the web gui under DNS… hopefully this is finally over.

Now with this in place, here is the query again

notice we also allow the trusted acl recursion
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
<code>dig ANY isc.org @x.x.x.x +edns=0

; &lt;&lt;&gt;&gt; DiG 9.8.1-P1 &lt;&lt;&gt;&gt; ANY isc.org @x.x.x.x +edns=0
;; global options: +cmd
;; Got answer:
;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: REFUSED, id: 53084
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;isc.org.                       IN      ANY

;; Query time: 50 msec
;; SERVER: x.x.x.x#53(x.x.x.x)
;; WHEN: Sat Apr 20 18:08:19 2013
;; MSG SIZE  rcvd: 36
</code>
nslookup tool… I know
1
2
3
4
5
<code>$ nslookup isc.org x.x.x.x
Server:         x.x.x.x
Address:        x.x.x.x#53

** server can't find isc.org: REFUSED
</code>

There you have it.

Category: BIND, NETWORKING | Los comentarios están deshabilitados en DNS Amplification DDoS Attack – ISC BIND
Marzo 21

Response Rate Limiting in the Domain Name System (DNS RRL)

This page describes DNS Response Rate Limiting (DNS RRL) which is an advanced pre-standard feature for domain name servers including CZ-NIC Knot DNS, NLNetLabs NSD, and ISC BIND9.

These patches and instructions pertain to authority name servers or authoritative views. Use of this kind of rate limiting for recursive or hybrid servers or views is currently unspecified.

Note Well:This is DNS RRL, meant to be implemented in many different name servers, it is not a BIND specific feature even though BIND was the first name server for which DNS RRL was implemented. DNS RRL will eventually be submitted to the IETF for standardization work. The need for DNS RRL is immediate and pressing, and the IETF processing of this work was therefore planned to come last rather than first.

References:

 

Instructions for BIND9:

  • Install the patched BIND9 server (BIND9 9.10 or later)
  • Read the above-linked technical note to familiarize yourself with the technology
  • Add something like this to your options block in your authoritative-only views:
        
    rate-limit {
        responses-per-second 15;
        window 5;
    };
  • Restart your name server and verify that it is operating normally
  • Test the rate limiting using a command similar to the one below:
    repeat 10 dig @server-ip-address +short +tries=1 +time=1 your-zone.com A

     

    The result should be at least one timeout out of the ten successive answers.

 

If you want to have a repeat function, you could add something like this to your ~/.bashrc:

<code>function repeat() { 
    local times="$1"; 
    shift; 
    local cmd="$@"; 

    for ((i = 1; i &lt;= $times; i++ )); do 
       eval "$cmd"; 
    done 
 }</code>

Source your ~/.bashrc again with . ~/.bashrc and you can call it:

<code> $ repeat 2 date
Mon Dec 21 14:25:50 CET 2009
Mon Dec 21 14:25:50 CET 2009

 $ repeat 3 echo "my name is $USER"
my name is raphink
my name is raphink
my name is raphink</code>

 

Category: BIND, NETWORKING | Los comentarios están deshabilitados en Response Rate Limiting in the Domain Name System (DNS RRL)
Marzo 21

How to block DNS Amplification Attack isc.org any attack

I’ve been seeing the DNS ANY attack against my DNS servers. I’m using PDNS but this should work with BIND or any DNS server. My DNS servers are not recursive, but they are attacking me anyway. I guess I’m the DDOS amplifier.

If you think the same thing is happening to you you can detect the attack as follows:

tcpdump -n udp dst port 53|grep ANY

If you are being attacked you will see something like this:

07:32:20.736483 IP 95.130.170.57.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:20.744356 IP 203.217.178.77.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:20.884926 IP 95.130.170.57.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:20.932583 IP 80.241.214.70.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:20.975589 IP 203.217.178.77.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:20.991550 IP 95.130.170.57.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.207135 IP 203.217.178.77.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.349899 IP 95.130.170.57.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.399836 IP 85.25.119.152.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.432679 IP 203.217.178.77.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.454784 IP 95.130.170.57.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.605888 IP 80.241.214.70.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.663643 IP 203.217.178.77.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.884477 IP 85.25.119.152.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)
07:32:21.895233 IP 203.217.178.77.25345 > 184.105.182.241.domain: 10809+ [1au] ANY? isc.org. (36)

After hunting around on the internet I found some code that actually blocks it. This code works:

iptables -v -I INPUT 1 -p udp –dport 53 -m string –from 50 –algo bm –hex-string ‘|0000FF0001|’ -m recent –set –name dnsanyquery

iptables -v -I INPUT 2 -p udp –dport 53 -m string –from 50 –algo bm –hex-string ‘|0000FF0001|’ -m recent –name dnsanyquery –rcheck –seconds 10 –hitcount 3 -j DROP

This allows two ANY queries from an IP over 10 seconds. You can adjust it any way you want. The above code inserts these rules into the first 2. You might also want to just use append and put these in early in the list.

iptables -v -A INPUT -p udp –dport 53 -m string –from 50 –algo bm –hex-string ‘|0000FF0001|’ -m recent –set –name dnsanyquery

iptables -v -A INPUT -p udp –dport 53 -m string –from 50 –algo bm –hex-string ‘|0000FF0001|’ -m recent –name dnsanyquery –rcheck –seconds 10 –hitcount 3 -j DROP

 Is it working? You can test it as follows:

dig @dns.yourdomain.com isc.org ANY

It should word 2 times in a row and fail on the 3rd try. This will allow some diagnostic ANY queries but block the high volume DDOS attacks.

I also have some recursive name servers and I use them heavilly internally. But I also use them lightly externally. These name servers are still open to the world but I threw some rate limiting on them for external use. That way they can be open and yet not abused.

First you have to allow your own network full access:

iptables -v -A INPUT -t filter -s 192.168.1.0/24 -j ACCEPT

Then you rate limit other addresses:

iptables -v -A INPUT -p udp –dport 53 -m recent –set –name dnsanyquery
iptables -v -A I

NPUT -p udp –dport 53 -m recent –name dnsanyquery –rcheck –seconds 1 –hitcount 10 -j DROP

 

Useful to detect DNS amplification

tcpdump -nnni eth1.2011 <strong>-c 100</strong> -w sample.txt dst port 53

Find out which bogus DNS resolvers are sending you an amplified attack.

awk '{print $3}' sample.txt | cut -d '.' -f1-4 | sort | uniq -c | sort -nr

 

Here’s I’m allowing 10 hits per second which is far less than an attacker would send.

&lt;code&gt;tcpdump -A -n dst port 53 and not host &amp;lt;local IP address&amp;gt;
tcpdump -n -i eth1.2011 udp port 53 and dst &amp;lt; local IP Address&amp;gt;

<strong>TEST</strong>
dig ANY isc.org @x.x.x.x +edns=0
Category: NETWORKING | Los comentarios están deshabilitados en How to block DNS Amplification Attack isc.org any attack
Marzo 20

Firewall HA with conntrackd and keepalived

As mentioned earlier, let’s see how to add HA to a linux/iptables-based firewall by means of keepalived and conntrackd.

There are a few scenarios for firewall HA. Probably, the most common one is the “classic” active-backup case where, at any time, one firewall is active and manages traffic, and the other is a “hot standby” ready to take over if the active one fails. In principle, since all the tools we’re going to use can communicate over multicast, it should be possible to extend the setup described here to more than two firewalls.

We’re going to assume this network setup:

firewallha

The two firewalls have a dedicated link (interface eth2 on both machines) to exchange session table synchronization messages, which is the recommended setup. If that is not possible, another interface can be used (for example, the internal LAN interface eth1). In that case, the configuration shown below should be adapted accordingly (essentially, use eth1 and 172.16.0.x instead of eth2 and 10.0.0.x, where x varies depending on the firewall). However beware that the recommendation of using a dedicated link exists for a reason: conntrackd can produce a lot of traffic. On a moderately busy firewall (about 33K connections on average), a quick test showed up to 1.6 Mbit/s of conntrackd synchronization traffic between the firewalls.

keepalived

The basic idea is that keepalived manages the failover of the (virtual) IPs using the standard VRRP protocol: at any time, the firewall that owns the virtual IPs replies to ARP requests (neighbor solicitiations for IPv6) and thus receives the traffic (this is accomplished by sending gratuitous ARPs for IPv4 and “gratuitous” neighbor advertisements for IPv6 when the firewall becomes active. Any HA product that has to move IPs uses this method).

Since the VRRP protocol performs failover of the virtual IPs, one may think that it’s all that we need to get HA. For some applications, this may be true; however, in the case of stateful firewalls a crude VRRP-only failover would disrupt existing sessions. The keyword here is stateful, that is, the firewall keeps a table of active sessions with various pieces of metadata about each one. When a previously idle firewall becomes active, it suddenly starts receiving packets belonging to established sessions, which however it knows nothing about. Thus, it would kill them, or try to handle the packets locally; in all cases, sessions would be disrupted. (We will see later that this problem can still occur for short times even when using conntrackd, but can be easily solved). For small setups it may be relatively fine, but if the firewall is a busy one the failover can kill hundreds of sessions. If we’re serious about HA, VRRP alone is not enough; the connection tracking table has to be kept in sync among firewalls, and this is where conntrackd comes into play.

conntrackd

Conntrackd is a complex tool. It can be used to collect traffic statistics on the firewalls, but also (and this is what we want here) to keep the stateful session table synchronized between the firewalls, so at any time they have the same information. Session information can be exchanged using a few different ways; here we’re going to use the recommended method (called FTFW) which uses a reliable messaging protocol. In turn, FTFW can use multicast or unicast UDP as its transport; here we’re using unicast. The sample configuration files that come with conntrackd have comments that explain how to set up multicast UDP if one wants to.

By default, there are two locations where session information is stored: the so-called internal cache is where the firewall stores its local session table (ie, sessions for which it’s passing traffic; this is a (partial) copy of the kernel session table, which can be inspected with tools like conntrack – without the trailing d); then, the external cache is where the firewall stores sessions it learns from the other firewall(s). During normal operation, the firewalls continuously exchange messages to inform the peer(s) about each one’s session table and its changes, so at any time each firewall knows its own and the other firewall’s sessions. When using two firewalls, one firewall’s internal cache should match the other’s external one, and viceversa.

When a firewall becomes active following a failover, it invokes a script that commits the external cache into the kernel table, and then resyncs the internal cache using the kernel table as the origin; the result is that from that moment on the firewall can start managing sessions for which it had not seen a single packet until then, just as if it had been managing them from their beginning. This is much better than what we would get if using only pure VRRP failover.
The commit script is invoked by keepalived when it detects that the firewall is changing state. The script is called primary-backup.sh and comes with conntrackd; most distributions put it into the documentation directory (eg /usr/share/doc/conntrackd or similar). The same script is invoked upon any state change (when the firewall becomes active, backup, or fails); it knows what happened because it’s passed a different argument for any possible state.

Note that it is also possible to disable the external cache (see the DisableExternalCache configuration directive). This way, all the sessions (local and learned) will always be stored directly into the kernel table/internal cache. This means that nothing needs to be done upon failover (or at most, only resyncing the internal cache with the kernel table), as the information the firewall needs to take over is already where it should be (the internal cache). So one may wonder why bother with the external cache at all; the official documentation mentions efficiency and resource usage concerns. Personally, using the external cache seems to work fairly well, so I didn’t have the need to mess about and disable it.

Run the commands listed below to copy the sample config file and failover script to the default directory for conntrackd, /etc/conntrackd/conntrackd.conf:

root@lj-fw-1:/# cd /usr/share/doc/conntrackd/examples/sync
root@lj-fw-1:/# gunzip ftfw/conntrackd.conf.gz
root@lj-fw-1:/# cp ftfw/conntrackd.conf /etc/conntrackd/
root@lj-fw-1:/# cp primary-backup.sh /etc/conntrackd

Configuration files

Here are the configuration files used for the scenario described here. keepalived.conf:

lobal_defs {
 notification_email {
 hxxx@xxx.cl # Email address for notifications
 }
 notification_email_from xxx@uni.cl # The from address for the n$
 smtp_server 217.7.1.120 # You can specifiy your own smtp server here
 smtp_connect_timeout 15
 router_id FW-A
}
vrrp_sync_group G1 {
 group {
 E1
 I1
 }
 notify_master "/etc/conntrackd/primary-backup.sh primary"
 notify_backup "/etc/conntrackd/primary-backup.sh backup"
 notify_fault "/etc/conntrackd/primary-backup.sh fault"
}
vrrp_instance E1 {
 interface eth0
 state MASTER
 nopreempt
 virtual_router_id 61
 priority 100
 advert_int 1
 authentication {
 auth_type PASS
 auth_pass zzzz
 }
 virtual_ipaddress {
 172.31.4.151/24 dev eth0 label eth0:0
 }
 garp_master_delay 5
}
vrrp_instance I1 {
 interface eth1
 state MASTER
 nopreempt
 virtual_router_id 62
 priority 100
 advert_int 1
 authentication {
 auth_type PASS
 auth_pass zzzz
 }
 virtual_ipaddress {
 172.16.10.100/24 dev eth1 label eth1:0
 }
 garp_master_delay 5
}

FW-B

global_defs {
 notification_email {
 hxxqs@cdcd.cl # Email address for notifications
 }
 notification_email_from xxx@uni.cl # The from address for the n$
 smtp_server 227.1.13.144 # You can specifiy your own smtp server here
 smtp_connect_timeout 15
 router_id FW-B
}
vrrp_sync_group G1 {
 group {
 E1
 I1
 }
 notify_master "/etc/conntrackd/primary-backup.sh primary"
 notify_backup "/etc/conntrackd/primary-backup.sh backup"
 notify_fault "/etc/conntrackd/primary-backup.sh fault"
}
vrrp_instance E1 {
 interface eth0
 state BACKUP
 nopreempt
 virtual_router_id 61
 priority 50
 advert_int 1
 authentication {
 auth_type PASS
 auth_pass zzzz
 }
 virtual_ipaddress {
 172.31.4.151/24 dev eth0 label eth0:0
 }
 garp_master_delay 5
}
vrrp_instance I1 {
 interface eth1
 state BACKUP
 nopreempt
 virtual_router_id 62
 priority 50
 advert_int 1
 authentication {
 auth_type PASS
 auth_pass zzzz
 }
 virtual_ipaddress {
 172.16.10.100/24 dev eth1 label eth1:0
 }
 nopreempt
 garp_master_delay 5
}

 

The above is from fw1; on fw2 it’s the same but the priority of each instance is 50 instead of 100.

conntrackd.conf (comments removed):

FW-A

Sync {
 Mode FTFW {
 DisableExternalCache Off
 CommitTimeout 1800
 PurgeTimeout 5
 }
UDP {
 IPv4_address 10.0.0.1
 IPv4_Destination_Address 10.0.0.2
 Port 3780
 Interface eth2
 SndSocketBuffer 1249280
 RcvSocketBuffer 1249280
 Checksum on
 }
}
General {
 Nice -20
 HashSize 32768
 HashLimit 131072
 LogFile on
 Syslog on
 LockFile /var/lock/conntrack.lock
 UNIX {
 Path /var/run/conntrackd.ctl
 Backlog 20
 }
 NetlinkBufferSize 2097152
 NetlinkBufferSizeMaxGrowth 8388608
 Filter From Userspace {
 Protocol Accept {
 TCP
 UDP
 ICMP # This requires a Linux kernel >= 2.6.31
 }
 Address Ignore {
 IPv4_address 127.0.0.1 # loopback
 IPv4_address 10.0.0.1
 IPv4_address 10.0.0.2
 IPv4_address 172.16.10.100
 IPv4_address 172.16.10.101
 IPv4_address 172.16.10.102
 IPv4_address 172.31.4.150
 IPv4_address 172.31.4.151
 IPv4_address 172.31.4.152
 }
 }
}

FW-B

Sync {
 Mode FTFW {
 DisableExternalCache Off
 CommitTimeout 1800
 PurgeTimeout 5
 }
UDP {
 IPv4_address 10.0.0.2
 IPv4_Destination_Address 10.0.0.1
 Port 3780
 Interface eth2
 SndSocketBuffer 1249280
 RcvSocketBuffer 1249280
 Checksum on
 }
}
General {
 Nice -20
 HashSize 32768
 HashLimit 131072
 LogFile on
 Syslog on
 LockFile /var/lock/conntrack.lock
 UNIX {
 Path /var/run/conntrackd.ctl
 Backlog 20
 }
 NetlinkBufferSize 2097152
 NetlinkBufferSizeMaxGrowth 8388608
 Filter From Userspace {
 Protocol Accept {
 TCP
 UDP
 ICMP # This requires a Linux kernel >= 2.6.31
 }
 Address Ignore {
 IPv4_address 127.0.0.1 # loopback
 IPv4_address 10.0.0.1
 IPv4_address 10.0.0.2
 IPv4_address 172.16.10.100
 IPv4_address 172.16.10.101
 IPv4_address 172.16.10.102
 IPv4_address 172.31.4.150
 IPv4_address 172.31.4.151
 IPv4_address 172.31.4.152
 }
 }
}

 

Again, the above is taken from fw1; on fw2, the UDP section has the source/destination IP addresses inverted.

The “Address Ignore” block should list ALL the IPs the firewall has (or can have) on local interfaces, including the VIPs. It doesn’t hurt to include some extra IP (eg those of the other firewall).

The “well-formed ruleset”

(Just in case you’re testing with everything set to ACCEPT and it doesn’t work)

One thing that is mentioned in the documentation but imho not stressed enough is the fact that the firewall MUST have what they call a “well-formed ruleset”, which essentially means that the firewall must DROP (not accept nor reject) any packet it doesn’t know about. It’s explained better in this email from the netfilter mailing list.

We briefly touched the issue earlier; even with conntrackd, it may still happen that during a failover the firewall that is becoming active receives some packet related to a session it doesn’t yet know about (eg. because failover isn’t instantaneous and the firewall hasn’t finished committing the external cache); under normal conditions, the firewall’s local TCP/IP stack may try to process such packets, resulting in potential disruption since it would almost certainly end up sending TCP RST or ICMP errors to one or both connection parties. One case is especially critical, it goes like this: an internal client is receiving data (eg downloading) from an external server, a failover happens, some of the packets the server is sending hit the firewall that is becoming active, which isn’t fully synced yet, so it sends RST to the server. Result: the server closes its side, but the client in the LAN still thinks the connection is valid, and hangs waiting for data. If it’s the client that gets the RST, what happens depends on the specific application; it may exit, or retry.

The moral of the story thus is that, for the failover to be seamless, it’s critical that the firewall ignore (drop, not reject) packets it doesn’t know about. In particular, a packet coming from the outside belonging to a NATed connection looks just like a packet addressed to the firewall, if the firewall has no state for the connection; so those packets have to be DROPped in the INPUT chain. In practice, this probably means a default DROP policy for the INPUT chain (ok, being a firewall it probably does it anyway, but better be explicit). Similarly, a DROP policy for the FORWARD chains will also help.

All this works because if the firewall drops unknown traffic, TCP or whatever protocol the communicating parties are using will notice the loss and sort it out (eg by retrasmitting packets).

Testing

So for example we can download some Debian DVD on two or more clients, to keep them busy with a long-running TCP connection:

wget -O/dev/null 'http://ftp.acc.umu.se/debian-cd/7.5.0/amd64/iso-dvd/debian-update-7.5.0-amd64-DVD-1.iso'

Open some other less intensive task, like ssh or telnet sessions, and perhaps watch some Internet video. In short, create many connections to the Internet through the active firewall. Once all this is in place, log into the active firewall (the one that has the VIPs), and stop or restart keepalived, to force a failover to the other firewall (if you stop keepalived, remember to start it again later before doing further tests). If everything is set up correctly, the VIPs should move to the other box and the active sessions in the LAN should keep working flawlessly. That’s it! For more thorough testing, the failover process can be repeated many times (within reason), and every time it should be transparent to clients.

Here’s a script that forces a failover between fw1 and fw2 and viceversa every N seconds, where N is a random number between 61 and 120 (of course, this is just for testing purposes):

#!/bin/bash
declare -a m
fws=( 172.16.10.101 172.16.10.102 )
i=0
maxi=$(( ${#fws[@]} - 1 ))
while true; do
 [ $i -gt $maxi ] && i=0
 fw=${fws[$i]}
#echo "deactivating $fw"
 ssh root@"${fw}" '/etc/init.d/keepalived restart'
# interval between 61 and 120 seconds
 period=$(($RANDOM % 60 + 61))
 #echo "sleeping $period seconds..."
 sleep $period
((i++))
done
Category: KEEPALIVED, NETWORKING | Los comentarios están deshabilitados en Firewall HA with conntrackd and keepalived
Marzo 6

How To Shrink a VMDK with ESXi5

I’ve had the luxury of working quite a bit with ESXi5 and how it relates to storage. With ESXi5, you can easily expand your virtual disks (VMDK file) for the VM on the fly, however there isn’t a simple way to reduce the size of your drives. As I’ve learned, best practices for VM’s is to always start small, as you can always increase later on with ease. Unfortunately, the previous administrator didn’t honor this, and started everything out with 600gb drives, and only utilizing 25gb. Needless to say, my SAN was getting chewed up by VMDK’s that only was using a fraction of the space allocated for it. After some research, this was the method that I use to reduce the size of the VMDK.

I should probably preface all of this that this is a dangerous procedure, and you run the risk of hosing your VM, so proceed at your own risk.

Most of my VM’s are windows machines, so I first need to reduce the windows partition. First I’ll need to defrag the drive. Then, open up the Disk Management via the MMC console snap-in, and right click on the drive, and select Shrink. It will tell you the maximum amount that it can shrink. If you want it to shrink more, it means that certain files are locked towards the end of the disk. In my case, I was able to delete/move the user profiles off of the disk, and that’s what was preventing me from reducing the disk to the size I wanted.

Great, so in this case, I was able to reduce a 600gb disk to 38gb.

Now I need to power down the VM, and SSH into the host and copy the VMDK file to make a backup of it. Hold onto these in case something goes wrong:

1 cp vmname.vmdk vmname-original.vmdk
2 cp vmname-flat.vmdk vmname-original-flat.vmdk

Now let’s open the vmdk within vi and modify the expected size. Towards the top, you will see a line that appears similar to the following:

1 # Extent description
2 RW 1258291200 VMFS “vmname-flat.vmdk”

The number value will need to be changed to the desired size using the following formula (x = desired size in GB):
vmdk_size = [x * (1024*1024*1024)] / 512

Because I like round numbers, I decided to make my new drive size of 40gb, so my new Extent description was as follows:

1 # Extent description
2 RW 83886080 VMFS “vmname-flat.vmdk”

Now I need to clone the drive to get it to the new size:

1 vmkfstools -i vmname.vmdk vmname-new-size.vmdk

Assuming all goes well, I will new delete the original (because I already made a copy of it just in case), and clone the disk to the original file name:

1 rm vmname.vmdk
2 rm vmname-flat.vmdk
3 vmkfstools -i vmname-new-size.vmdk vmname.vmdk

Now I should be able to start the VM again, and the new disk size will be shown!

– See more at: http://www.whitneytechnologies.com/?p=270#sthash.xXOHxkXs.dpuf

Category: VMWARE | Los comentarios están deshabilitados en How To Shrink a VMDK with ESXi5