Hi Folks, This may be a Matthew specific question - I'm testing out the patch he kindly posted to me on uknof at the beginning of June to enable relaying to a different UDP port. Early disclaimer: I'm testing this on a Cumulus Linux box - whilst they are Debian underneath, there are some 'Cumulus-isms' on the network side that potentially might be in play here. I am using freeradius_head_v3-2-x (downloaded 12/06/2022) with these patches applied: https://github.com/mcnewton/freeradius-server/commit/23f1856611722e30c997969... For testing, I have a Cumulus box with two interfaces that we care about: vlan4091 which has a test client connected to it, and vlan545 which is the connection back to the rest of the network. Our test DHCP server (217.198.54.101) is in this "rest of the network" - IP routing all works as expected, and you can ping the DHCP server from the vlan4091 interface IP address (217.198.52.2) on the Cumulus router, and from clients on that VLAN - and vice versa. When I send in a test DHCP request from a client connected to vlan4091, FR receives it and and re-sends it out to 217.198.54.101 on port 1067. However, the DHCP server never sees that packet. After some rummaging with strace and tcpdump, I've determined that FR is transmitting the relayed packet out of vlan4091 (I saw suspect looking ARPs, and put in a static ARP entry after which I see the frame transmitted on vlan4091) and not following the default route up out of vlan545. Looking in proto_dhcp/dhcpd.c at dhcprelay_process_client_request, there's nothing in there to suggest that the outgoing interface is set and routing is overridden: But it looks like some parts are copied over from the socket handling the client communication; is there something here that could cause what I'm seeing? My other thought that if this was a general problem it would break FR DHCP relaying in a lot of people's use cases - and there aren't lots of people grumbling about this, so there must be something wacky going on with my setup but I'm slightly at a loss to see what's happening. Any pointers greatfully received. Paul. # radiusd -X FreeRADIUS Version 3.2.1 Copyright (C) 1999-2021 The FreeRADIUS server project and contributors There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE You may redistribute copies of FreeRADIUS under the terms of the GNU General Public License For more information about these matters, see the file named COPYRIGHT Starting - reading configuration files ... including dictionary file /usr/local/share/freeradius/dictionary including dictionary file /usr/local/share/freeradius/dictionary.dhcp including dictionary file /usr/local/share/freeradius/dictionary.vqp including dictionary file /usr/local/etc/raddb/dictionary including configuration file /usr/local/etc/raddb/radiusd.conf including configuration file /usr/local/etc/raddb/proxy.conf including configuration file /usr/local/etc/raddb/clients.conf including files in directory /usr/local/etc/raddb/mods-enabled/ including configuration file /usr/local/etc/raddb/mods-enabled/attr_filter including configuration file /usr/local/etc/raddb/mods-enabled/date including configuration file /usr/local/etc/raddb/mods-enabled/preprocess including configuration file /usr/local/etc/raddb/mods-enabled/exec including configuration file /usr/local/etc/raddb/mods-enabled/detail including configuration file /usr/local/etc/raddb/mods-enabled/echo including configuration file /usr/local/etc/raddb/mods-enabled/always including configuration file /usr/local/etc/raddb/mods-enabled/expr including configuration file /usr/local/etc/raddb/mods-enabled/dhcp including configuration file /usr/local/etc/raddb/mods-enabled/digest including configuration file /usr/local/etc/raddb/mods-enabled/linelog including files in directory /usr/local/etc/raddb/policy.d/ including configuration file /usr/local/etc/raddb/policy.d/moonshot-targeted-ids including configuration file /usr/local/etc/raddb/policy.d/debug including configuration file /usr/local/etc/raddb/policy.d/operator-name including configuration file /usr/local/etc/raddb/policy.d/canonicalization including configuration file /usr/local/etc/raddb/policy.d/control including configuration file /usr/local/etc/raddb/policy.d/cui including configuration file /usr/local/etc/raddb/policy.d/abfab-tr including configuration file /usr/local/etc/raddb/policy.d/filter including configuration file /usr/local/etc/raddb/policy.d/dhcp including configuration file /usr/local/etc/raddb/policy.d/accounting including configuration file /usr/local/etc/raddb/policy.d/eap including configuration file /usr/local/etc/raddb/policy.d/rfc7542 including files in directory /usr/local/etc/raddb/sites-enabled/ including configuration file /usr/local/etc/raddb/sites-enabled/test_dhcp_relay main { security { allow_core_dumps = no } name = "radiusd" prefix = "/usr/local" localstatedir = "/usr/local/var" logdir = "/usr/local/var/log/radius" run_dir = "/usr/local/var/run/radiusd" } main { name = "radiusd" prefix = "/usr/local" localstatedir = "/usr/local/var" sbindir = "/usr/local/sbin" logdir = "/usr/local/var/log/radius" run_dir = "/usr/local/var/run/radiusd" libdir = "/usr/local/lib" radacctdir = "/usr/local/var/log/radius/radacct" hostname_lookups = no max_request_time = 30 cleanup_delay = 5 max_requests = 16384 postauth_client_lost = no pidfile = "/usr/local/var/run/radiusd/radiusd.pid" checkrad = "/usr/local/sbin/checkrad" debug_level = 0 proxy_requests = yes log { stripped_names = no auth = no auth_badpass = no auth_goodpass = no colourise = yes msg_denied = "You are already logged in - access denied" } resources { } security { max_attributes = 200 reject_delay = 1.000000 status_server = yes allow_vulnerable_openssl = "no" } } radiusd: #### Loading Realms and Home Servers #### proxy server { retry_delay = 5 retry_count = 3 default_fallback = no dead_time = 120 wake_all_if_all_dead = no } home_server localhost { ipaddr = 127.0.0.1 port = 1812 type = "auth" secret = <<< secret >>> response_window = 20.000000 response_timeouts = 1 max_outstanding = 65536 zombie_period = 40 status_check = "status-server" ping_interval = 30 check_interval = 30 check_timeout = 4 num_answers_to_alive = 3 revive_interval = 120 limit { max_connections = 16 max_requests = 0 lifetime = 0 idle_timeout = 0 } coa { irt = 2 mrt = 16 mrc = 5 mrd = 30 } } home_server_pool my_auth_failover { type = fail-over home_server = localhost } realm example.com { auth_pool = my_auth_failover } realm LOCAL { } radiusd: #### Loading Clients #### client localhost { ipaddr = 127.0.0.1 require_message_authenticator = no secret = <<< secret >>> nas_type = "other" proto = "*" limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } client localhost_ipv6 { ipv6addr = ::1 require_message_authenticator = no secret = <<< secret >>> limit { max_connections = 16 lifetime = 0 idle_timeout = 30 } } Debugger not attached radiusd: #### Instantiating modules #### modules { # Loaded module rlm_attr_filter # Loading module "attr_filter.post-proxy" from file /usr/local/etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.post-proxy { filename = "/usr/local/etc/raddb/mods-config/attr_filter/post-proxy" key = "%{Realm}" relaxed = no } # Loading module "attr_filter.pre-proxy" from file /usr/local/etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.pre-proxy { filename = "/usr/local/etc/raddb/mods-config/attr_filter/pre-proxy" key = "%{Realm}" relaxed = no } # Loading module "attr_filter.access_reject" from file /usr/local/etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.access_reject { filename = "/usr/local/etc/raddb/mods-config/attr_filter/access_reject" key = "%{User-Name}" relaxed = no } # Loading module "attr_filter.access_challenge" from file /usr/local/etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.access_challenge { filename = "/usr/local/etc/raddb/mods-config/attr_filter/access_challenge" key = "%{User-Name}" relaxed = no } # Loading module "attr_filter.accounting_response" from file /usr/local/etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.accounting_response { filename = "/usr/local/etc/raddb/mods-config/attr_filter/accounting_response" key = "%{User-Name}" relaxed = no } # Loading module "attr_filter.coa" from file /usr/local/etc/raddb/mods-enabled/attr_filter attr_filter attr_filter.coa { filename = "/usr/local/etc/raddb/mods-config/attr_filter/coa" key = "%{User-Name}" relaxed = no } # Loaded module rlm_date # Loading module "date" from file /usr/local/etc/raddb/mods-enabled/date date { format = "%b %e %Y %H:%M:%S %Z" utc = no } # Loading module "wispr2date" from file /usr/local/etc/raddb/mods-enabled/date date wispr2date { format = "%Y-%m-%dT%H:%M:%S" utc = no } # Loaded module rlm_preprocess # Loading module "preprocess" from file /usr/local/etc/raddb/mods-enabled/preprocess preprocess { huntgroups = "/usr/local/etc/raddb/mods-config/preprocess/huntgroups" hints = "/usr/local/etc/raddb/mods-config/preprocess/hints" with_ascend_hack = no ascend_channels_per_line = 23 with_ntdomain_hack = no with_specialix_jetstream_hack = no with_cisco_vsa_hack = no with_alvarion_vsa_hack = no } # Loaded module rlm_exec # Loading module "exec" from file /usr/local/etc/raddb/mods-enabled/exec exec { wait = no input_pairs = "request" shell_escape = yes timeout = 10 } # Loaded module rlm_detail # Loading module "detail" from file /usr/local/etc/raddb/mods-enabled/detail detail { filename = "/usr/local/var/log/radius/radacct/%{%{Packet-Src-IP-Address}:-%{Packet-Src-IPv6-Address}}/detail-%Y%m%d" header = "%t" permissions = 384 locking = no escape_filenames = no log_packet_header = no } # Loading module "echo" from file /usr/local/etc/raddb/mods-enabled/echo exec echo { wait = yes program = "/bin/echo %{User-Name}" input_pairs = "request" output_pairs = "reply" shell_escape = yes } # Loaded module rlm_always # Loading module "reject" from file /usr/local/etc/raddb/mods-enabled/always always reject { rcode = "reject" simulcount = 0 mpp = no } # Loading module "fail" from file /usr/local/etc/raddb/mods-enabled/always always fail { rcode = "fail" simulcount = 0 mpp = no } # Loading module "ok" from file /usr/local/etc/raddb/mods-enabled/always always ok { rcode = "ok" simulcount = 0 mpp = no } # Loading module "handled" from file /usr/local/etc/raddb/mods-enabled/always always handled { rcode = "handled" simulcount = 0 mpp = no } # Loading module "invalid" from file /usr/local/etc/raddb/mods-enabled/always always invalid { rcode = "invalid" simulcount = 0 mpp = no } # Loading module "userlock" from file /usr/local/etc/raddb/mods-enabled/always always userlock { rcode = "userlock" simulcount = 0 mpp = no } # Loading module "notfound" from file /usr/local/etc/raddb/mods-enabled/always always notfound { rcode = "notfound" simulcount = 0 mpp = no } # Loading module "noop" from file /usr/local/etc/raddb/mods-enabled/always always noop { rcode = "noop" simulcount = 0 mpp = no } # Loading module "updated" from file /usr/local/etc/raddb/mods-enabled/always always updated { rcode = "updated" simulcount = 0 mpp = no } # Loaded module rlm_expr # Loading module "expr" from file /usr/local/etc/raddb/mods-enabled/expr expr { safe_characters = "@abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789.-_: /äéöüàâæçèéêëîïôœùûüaÿÄÉÖÜßÀÂÆÇÈÉÊËÎÏÔŒÙÛÜŸ" } # Loaded module rlm_dhcp # Loading module "dhcp" from file /usr/local/etc/raddb/mods-enabled/dhcp # Loaded module rlm_digest # Loading module "digest" from file /usr/local/etc/raddb/mods-enabled/digest # Loaded module rlm_linelog # Loading module "linelog" from file /usr/local/etc/raddb/mods-enabled/linelog linelog { filename = "/usr/local/var/log/radius/linelog" escape_filenames = no syslog_severity = "info" permissions = 384 format = "This is a log message for %{User-Name}" reference = "messages.%{%{reply:Packet-Type}:-default}" } # Loading module "log_accounting" from file /usr/local/etc/raddb/mods-enabled/linelog linelog log_accounting { filename = "/usr/local/var/log/radius/linelog-accounting" escape_filenames = no syslog_severity = "info" permissions = 384 format = "" reference = "Accounting-Request.%{%{Acct-Status-Type}:-unknown}" } instantiate { } # Instantiating module "attr_filter.post-proxy" from file /usr/local/etc/raddb/mods-enabled/attr_filter reading pairlist file /usr/local/etc/raddb/mods-config/attr_filter/post-proxy # Instantiating module "attr_filter.pre-proxy" from file /usr/local/etc/raddb/mods-enabled/attr_filter reading pairlist file /usr/local/etc/raddb/mods-config/attr_filter/pre-proxy # Instantiating module "attr_filter.access_reject" from file /usr/local/etc/raddb/mods-enabled/attr_filter reading pairlist file /usr/local/etc/raddb/mods-config/attr_filter/access_reject # Instantiating module "attr_filter.access_challenge" from file /usr/local/etc/raddb/mods-enabled/attr_filter reading pairlist file /usr/local/etc/raddb/mods-config/attr_filter/access_challenge # Instantiating module "attr_filter.accounting_response" from file /usr/local/etc/raddb/mods-enabled/attr_filter reading pairlist file /usr/local/etc/raddb/mods-config/attr_filter/accounting_response # Instantiating module "attr_filter.coa" from file /usr/local/etc/raddb/mods-enabled/attr_filter reading pairlist file /usr/local/etc/raddb/mods-config/attr_filter/coa # Instantiating module "preprocess" from file /usr/local/etc/raddb/mods-enabled/preprocess reading pairlist file /usr/local/etc/raddb/mods-config/preprocess/huntgroups reading pairlist file /usr/local/etc/raddb/mods-config/preprocess/hints # Instantiating module "detail" from file /usr/local/etc/raddb/mods-enabled/detail # Instantiating module "reject" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "fail" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "ok" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "handled" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "invalid" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "userlock" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "notfound" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "noop" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "updated" from file /usr/local/etc/raddb/mods-enabled/always # Instantiating module "linelog" from file /usr/local/etc/raddb/mods-enabled/linelog # Instantiating module "log_accounting" from file /usr/local/etc/raddb/mods-enabled/linelog } # modules radiusd: #### Loading Virtual Servers #### server { # from file /usr/local/etc/raddb/radiusd.conf } # server server dhcp.vlan4091 { # from file /usr/local/etc/raddb/sites-enabled/test_dhcp_relay # Loading dhcp DHCP-Discover {...} Compiling dhcp DHCP-Discover for attr DHCP-Message-Type # Loading dhcp DHCP-Request {...} Compiling dhcp DHCP-Request for attr DHCP-Message-Type } # server dhcp.vlan4091 radiusd: #### Opening IP addresses and Ports #### listen { type = "dhcp" ipaddr = * port = 67 } Listening on dhcp interface vlan4091 address * port 67 bound to server dhcp.vlan4091 Ready to process requests Received DHCP-Discover of Id 68879701 from 0.0.0.0:68 to 255.255.255.255:67 DHCP-Opcode = Client-Message DHCP-Hardware-Type = Ethernet DHCP-Hardware-Address-Length = 6 DHCP-Hop-Count = 0 DHCP-Transaction-Id = 1753716481 DHCP-Number-of-Seconds = 0 DHCP-Flags = 0 DHCP-Client-IP-Address = 0.0.0.0 DHCP-Your-IP-Address = 0.0.0.0 DHCP-Server-IP-Address = 0.0.0.0 DHCP-Gateway-IP-Address = 0.0.0.0 DHCP-Client-Hardware-Address = 90:e2:ba:86:3c:24 DHCP-Hostname = "test-optiplex" DHCP-Message-Type = DHCP-Discover DHCP-Parameter-Request-List = DHCP-Subnet-Mask DHCP-Parameter-Request-List = DHCP-Broadcast-Address DHCP-Parameter-Request-List = DHCP-Time-Offset DHCP-Parameter-Request-List = DHCP-Classless-Static-Route DHCP-Parameter-Request-List = DHCP-Router-Address DHCP-Parameter-Request-List = DHCP-Domain-Name DHCP-Parameter-Request-List = DHCP-Domain-Name-Server DHCP-Parameter-Request-List = DHCP-Hostname DHCP-Parameter-Request-List = DHCP-Domain-Search DHCP-Parameter-Request-List = DHCP-Interface-MTU-Size DHCP-Client-Identifier = 0x0190e2ba863c24 DHCP-Relay-Circuit-Id = 0x4c41423031206574682031372f31312f312f312f31 DHCP-Relay-Remote-Id = 0x6c616230315f6c66633030303030323233 DHCP-Network-Subnet = 0.0.0.0/32 (0) Received code 1025 Id 1753716481 from 0.0.0.0:68 to 255.255.255.255:67 length 548 (0) DHCP-Opcode = Client-Message (0) DHCP-Hardware-Type = Ethernet (0) DHCP-Hardware-Address-Length = 6 (0) DHCP-Hop-Count = 0 (0) DHCP-Transaction-Id = 1753716481 (0) DHCP-Number-of-Seconds = 0 (0) DHCP-Flags = 0 (0) DHCP-Client-IP-Address = 0.0.0.0 (0) DHCP-Your-IP-Address = 0.0.0.0 (0) DHCP-Server-IP-Address = 0.0.0.0 (0) DHCP-Gateway-IP-Address = 0.0.0.0 (0) DHCP-Client-Hardware-Address = 90:e2:ba:86:3c:24 (0) DHCP-Hostname = "test-optiplex" (0) DHCP-Message-Type = DHCP-Discover (0) DHCP-Parameter-Request-List = DHCP-Subnet-Mask (0) DHCP-Parameter-Request-List = DHCP-Broadcast-Address (0) DHCP-Parameter-Request-List = DHCP-Time-Offset (0) DHCP-Parameter-Request-List = DHCP-Classless-Static-Route (0) DHCP-Parameter-Request-List = DHCP-Router-Address (0) DHCP-Parameter-Request-List = DHCP-Domain-Name (0) DHCP-Parameter-Request-List = DHCP-Domain-Name-Server (0) DHCP-Parameter-Request-List = DHCP-Hostname (0) DHCP-Parameter-Request-List = DHCP-Domain-Search (0) DHCP-Parameter-Request-List = DHCP-Interface-MTU-Size (0) DHCP-Client-Identifier = 0x0190e2ba863c24 (0) DHCP-Relay-Circuit-Id = 0x4c41423031206574682031372f31312f312f312f31 (0) DHCP-Relay-Remote-Id = 0x6c616230315f6c66633030303030323233 (0) DHCP-Network-Subnet = 0.0.0.0/32 Trying sub-section dhcp DHCP-Discover {...} (0) dhcp DHCP-Discover { (0) update config { (0) &DHCP-Relay-To-IP-Address := 217.198.54.101 (0) &DHCP-Relay-To-Port := 1067 (0) } # update config = noop (0) update request { (0) &DHCP-Gateway-IP-Address := 217.198.52.2 (0) } # update request = noop (0) [ok] = ok (0) } # dhcp DHCP-Discover = ok DHCP-Opcode = Client-Message DHCP-Hardware-Type = Ethernet DHCP-Hardware-Address-Length = 6 DHCP-Hop-Count = 1 DHCP-Transaction-Id = 1753716481 DHCP-Number-of-Seconds = 0 DHCP-Flags = 0 DHCP-Client-IP-Address = 0.0.0.0 DHCP-Your-IP-Address = 0.0.0.0 DHCP-Server-IP-Address = 0.0.0.0 DHCP-Gateway-IP-Address = 217.198.52.2 DHCP-Client-Hardware-Address = 90:e2:ba:86:3c:24 DHCP-Server-Host-Name = "" DHCP-Boot-Filename = "" DHCP-Hostname = "test-optiplex" DHCP-Parameter-Request-List = DHCP-Subnet-Mask DHCP-Parameter-Request-List = DHCP-Broadcast-Address DHCP-Parameter-Request-List = DHCP-Time-Offset DHCP-Parameter-Request-List = DHCP-Classless-Static-Route DHCP-Parameter-Request-List = DHCP-Router-Address DHCP-Parameter-Request-List = DHCP-Domain-Name DHCP-Parameter-Request-List = DHCP-Domain-Name-Server DHCP-Parameter-Request-List = DHCP-Hostname DHCP-Parameter-Request-List = DHCP-Domain-Search DHCP-Parameter-Request-List = DHCP-Interface-MTU-Size DHCP-Client-Identifier = 0x0190e2ba863c24 DHCP-Relay-Circuit-Id = 0x4c41423031206574682031372f31312f312f312f31 DHCP-Relay-Remote-Id = 0x6c616230315f6c66633030303030323233 dhcprelay_process_client_request: paul: Here 1 Sending DHCP-Discover Id 68879701 from 0.0.0.0:67 to 217.198.54.101:1067 (0) Not sending reply to client. (0) Finished request (0) Cleaning up request packet ID 1753716481 with timestamp +5 due to done Ready to process requests Whilst this is going on, a tcpdump on vlan 4091 shows: # tcpdump -n -v -i vlan4091 -e -s 1500 tcpdump: listening on vlan4091, link-type EN10MB (Ethernet), capture size 1500 bytes 11:09:43.498585 90:e2:ba:86:3c:24 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 590: (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 576) 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 90:e2:ba:86:3c:24, length 548, xid 0x7ca972e9, Flags [none] Client-Ethernet-Address 90:e2:ba:86:3c:24 Vendor-rfc1048 Extensions Magic Cookie 0x63825363 Hostname Option 12, length 13: "test-optiplex" DHCP-Message Option 53, length 1: Discover Parameter-Request Option 55, length 10: Subnet-Mask, BR, Time-Zone, Classless-Static-Route Default-Gateway, Domain-Name, Domain-Name-Server, Hostname Option 119, MTU Client-ID Option 61, length 7: ether 90:e2:ba:86:3c:24 Agent-Information Option 82, length 42: Circuit-ID SubOption 1, length 21: LAB01 eth 17/11/1/1/1 Remote-ID SubOption 2, length 17: lab01_lfc00000223 11:09:43.501284 04:3f:72:50:0a:94 > 02:aa:bb:cc:dd:ee, ethertype IPv4 (0x0800), length 367: (tos 0x0, ttl 64, id 16145, offset 0, flags [none], proto UDP (17), length 353) 217.198.52.2.67 > 217.198.54.101.1067: BOOTP/DHCP, Request from 90:e2:ba:86:3c:24, length 325, hops 1, xid 0x7ca972e9, Flags [none] Gateway-IP 217.198.52.2 Client-Ethernet-Address 90:e2:ba:86:3c:24 Vendor-rfc1048 Extensions Magic Cookie 0x63825363 DHCP-Message Option 53, length 1: Discover Hostname Option 12, length 13: "test-optiplex" Parameter-Request Option 55, length 10: Subnet-Mask, BR, Time-Zone, Classless-Static-Route Default-Gateway, Domain-Name, Domain-Name-Server, Hostname Option 119, MTU Client-ID Option 61, length 7: ether 90:e2:ba:86:3c:24 Agent-Information Option 82, length 42: Circuit-ID SubOption 1, length 21: LAB01 eth 17/11/1/1/1 Remote-ID SubOption 2, length 17: lab01_lfc00000223 ^C The MAC 02:aa:bb:cc:dd:ee comes from the static ARP entry on interface vlan4091 for 217.198.54.101