{"id":291,"date":"2017-11-04T15:47:28","date_gmt":"2017-11-04T15:47:28","guid":{"rendered":"https:\/\/www.spotonoracle.com\/?p=291"},"modified":"2017-11-04T16:11:26","modified_gmt":"2017-11-04T16:11:26","slug":"12c-oracle-dcd-kernel-call-dive","status":"publish","type":"post","link":"https:\/\/www.spotonoracle.com\/?p=291","title":{"rendered":"12c Oracle DCD – kernel call dive"},"content":{"rendered":"

As you probably read in the 12c documentation Oracle has changed the mechanism for DCD (Dead Connection Detection) from sending TNS packets to using TCP KEEPALIVE on the socket. Unfortunately, the documentation is extremely sparse about implementation details:
\n\"\"<\/a>
\nWhen I was tracking down a firewall configuration issue recently I was in need of a few more details. Specifically, I wanted to know what TCP_KEEPIDLE, TCP_KEEPINTVL, and TCP_KEEPCNT are set to for various values of SQLNET.EXPIRE_TIME.
\nSince Oracle reads sqlnet.ora and sets the socket options early in the process startup (spawned by the listener) I opted for SystemTap to caputre all calls to “setsockopt”. Unfortunately, on my 7.4 Oracle Linux I ran into a stupid SystemTap bug. So, DTrace it is:<\/p>\n

\r\nsyscall::setsockopt:entry\r\n{\r\n  printf("%s | socket=%d | level=%d | optname=%d | optlen=%d", execname, arg0, arg1, arg2, arg4);\r\n  self->optval = *(int *)copyin(arg3, arg4);\r\n}\r\n\r\nsyscall::setsockopt:return\r\n{\r\n  printf("optval=%d\\n", self->optval);\r\n}\r\n<\/pre>\n
I ran it on dedicated lab VM to minimize cluttering of the output as the D script captures all “setsockopt” calls, not just the ones from Oracle.
\nBut, before I show the output we need to know a few things.<\/p>\n
The function definition<\/strong> for “setsockopt”:
\nThis is the kernel function to modify socket options like TCP_KEEPALIVE, etc.<\/p>\n
\r\nint setsockopt(\r\n  int sockfd,\r\n  int level,\r\n  int optname,\r\n  const void *optval,\r\n  socklen_t optlen);\r\n<\/pre>\n
\r\nsockfd  => file descriptor for the socket\r\nlevel   => the level on which to set the option\/value (socket or any protocol)\r\noptname => the number representing the option name according to header files\r\noptval  => the value to set for the given option\r\n<\/pre>\n
Note, the sequence of arguments matches arg0 .. arg4 in above D script.<\/p>\n
The relevant socket level & options<\/strong> (from socket.h):<\/p>\n
\r\n# level\r\n1 => SOL_SOCKET\r\n \r\n# options\r\n9  => SO_KEEPALIVE\r\n20 => SO_RCVTIMEO\r\n21 => SO_SNDTIMEO\r\n<\/pre>\n
The relevant TCP level & options<\/strong> (from tcp.h)<\/p>\n
\r\n# level\r\n6 => SOL_TCP\r\n \r\n# options\r\n1 => TCP_NODELAY\r\n4 => TCP_KEEPIDLE\r\n5 => TCP_KEEPINTVL\r\n6 => TCP_KEEPCNT\r\n<\/pre>\n
Now we’ve got all the information necessary to interpret the output from my D script.<\/p>\n
SQLNET.EXPIRE_TIME = 1<\/u><\/p>\n
\r\ndtrace: script 'sockopt.d' matched 2 probes\r\nCPU     ID                    FUNCTION:NAME\r\n  0    125                 setsockopt:entry tnslsnr | socket=14 | level=6 | optname=1 | optlen=4\r\n  0    126                setsockopt:return optval=1\r\n  \r\n  1    125                 setsockopt:entry oracle_7090_dev | socket=14 | level=6 | optname=1 | optlen=4\r\n  1    126                setsockopt:return optval=1\r\n  \r\n  1    125                 setsockopt:entry oracle_7090_dev | socket=14 | level=1 | optname=9 | optlen=4\r\n  1    126                setsockopt:return optval=1\r\n  \r\n  1    125                 setsockopt:entry oracle_7090_dev | socket=14 | level=6 | optname=4 | optlen=4\r\n  1    126                setsockopt:return optval=60\r\n  \r\n  1    125                 setsockopt:entry oracle_7090_dev | socket=14 | level=6 | optname=5 | optlen=4\r\n  1    126                setsockopt:return optval=6\r\n  \r\n  1    125                 setsockopt:entry oracle_7090_dev | socket=14 | level=6 | optname=6 | optlen=4\r\n  1    126                setsockopt:return optval=10\r\n<\/pre>\n
The output shows the following:<\/p>\n
    \n
  • TCP_KEEPALIVE is enabled<\/li>\n
  • TCP_KEEPIDLE  is set to the value of SQLNET.EXPIRE_TIME in seconds<\/li>\n
  • TCP_KEEPINTVL is set to 6<\/li>\n
  • TCP_KEEPCNT   is set to 10<\/li>\n<\/ul>\n
    OK, let’s gradually increase SQLNET.EXPIRE_TIME and see what happens.<\/p>\n
    SQLNET.EXPIRE_TIME = 10<\/u><\/p>\n
    \r\n... \r\n  0    125                 setsockopt:entry oracle_6400_dev | socket=14 | level=6 | optname=4 | optlen=4\r\n  0    126                setsockopt:return optval=600\r\n  \r\n  0    125                 setsockopt:entry oracle_6400_dev | socket=14 | level=6 | optname=5 | optlen=4\r\n  0    126                setsockopt:return optval=6\r\n  \r\n  0    125                 setsockopt:entry oracle_6400_dev | socket=14 | level=6 | optname=6 | optlen=4\r\n  0    126                setsockopt:return optval=10\r\n...\r\n<\/pre>\n
    SQLNET.EXPIRE_TIME = 60<\/u><\/p>\n
    \r\n...\r\n  1    125                 setsockopt:entry oracle_7147_dev | socket=14 | level=6 | optname=4 | optlen=4\r\n  1    126                setsockopt:return optval=3600\r\n  \r\n  1    125                 setsockopt:entry oracle_7147_dev | socket=14 | level=6 | optname=5 | optlen=4\r\n  1    126                setsockopt:return optval=6\r\n  \r\n  1    125                 setsockopt:entry oracle_7147_dev | socket=14 | level=6 | optname=6 | optlen=4\r\n  1    126                setsockopt:return optval=10\r\n...\r\n<\/pre>\n
    SQLNET.EXPIRE_TIME = 120<\/u><\/p>\n
    \r\n...\r\n  1    125                 setsockopt:entry oracle_7163_dev | socket=14 | level=6 | optname=4 | optlen=4\r\n  1    126                setsockopt:return optval=7200\r\n  \r\n  1    125                 setsockopt:entry oracle_7163_dev | socket=14 | level=6 | optname=5 | optlen=4\r\n  1    126                setsockopt:return optval=6\r\n  \r\n  1    125                 setsockopt:entry oracle_7163_dev | socket=14 | level=6 | optname=6 | optlen=4\r\n  1    126                setsockopt:return optval=10\r\n...\r\n<\/pre>\n
    Conclusion<\/strong>
    \nFrom this I conclude that Oracle always sets TCP_KEEPIDLE to the value of SQLNET.EXPIRE_TIME in seconds.
    \nTCP_KEEPCNT and TCP_KEEPCNT are not adjusted based on SQLNET.EXPIRE_TIME but are always set to 6 and 10, respectively.<\/p>\n
    For the sake of completeness, here’s what happens when the connection closes.<\/p>\n
     \r\n  1    125                 setsockopt:entry oracle_7090_dev | socket=14 | level=1 | optname=21 | optlen=16\r\n  1    126                setsockopt:return optval=0\r\n  \r\n  1    125                 setsockopt:entry oracle_7090_dev | socket=14 | level=1 | optname=20 | optlen=16\r\n  1    126                setsockopt:return optval=0\r\n  \r\n  1    125                 setsockopt:entry oracle_7090_dev | socket=14 | level=1 | optname=9 | optlen=4\r\n  1    126                setsockopt:return optval=0\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"
    As you probably read in the 12c documentation Oracle has changed the mechanism for DCD (Dead Connection Detection) from sending TNS packets to using TCP KEEPALIVE on the socket. Unfortunately, the documentation is extremely sparse about implementation details: When I was tracking down a firewall configuration issue recently I was in need of a few […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,16],"tags":[],"class_list":["post-291","post","type-post","status-publish","format-standard","hentry","category-internals","category-linux"],"_links":{"self":[{"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=\/wp\/v2\/posts\/291","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=291"}],"version-history":[{"count":4,"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=\/wp\/v2\/posts\/291\/revisions"}],"predecessor-version":[{"id":296,"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=\/wp\/v2\/posts\/291\/revisions\/296"}],"wp:attachment":[{"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=291"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=291"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.spotonoracle.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=291"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}