ChatGPT is getting a lot of attention. Usually, when I hear about ChatGPT, the context is generally around job security. Recently, in one of the team meetings, I got an interesting question: Where can we use ChatGPT to offload some of the Service Desk network troubleshooting tickets? So, I started exploring this subject.
For a simple test, I build a straightforward lab topology with two routers and IPSec VPN with OSPF running between them. Router configuration was implemented using old-style Cisco IPSec VTI syntax with IKEv1, which is currently not widely used. After the lab was built and working, I broke the router configuration to simulate network issues. For that, IKE phase II policies were changed. Performed changes, broke connectivity between routers, OSPF, and IPSec started generating error logs. Those logs were fed into ChatGPT to see how it can help speed up the troubleshooting process and see if it can exactly tell what the reason for the connectivity issue is.
Interaction with ChatGPT for troubleshooting network issues showed many accuracy complications. Most of them were connected to the problem context having many solutions. Still, those ways are not inter-compared, and you shall choose one or the other. If you started doing feature implementation the first way, you cannot finish or fix it the second way.
For example, on Cisco routers, there are at least ten different ways and syntaxes of IPSec VPN configuration, most of which are incompatible. You shall configure one or another on both ends of the IPSec VPN. Encryption/Authentication/Hashing capabilities at the data plane don’t change, but the implementation and syntax differ. The evolution of Cisco IOS also led to CLI syntax change.
The reason behind the inaccurate mixture of the problem solutions is the unfamiliarity with its contexts and the stochastic nature of the ChatGPT algorithm. Stochastic algorithms are a type of algorithm that use randomness to make decisions or solve problems. When ChatGPT doesn’t know the exact answer to raised questions, it tries to predict the answer pseudo-randomly.
As the issue was with the IPSec VPN configuration error, and ChatGPT did not know the exact answer, it looked at the context, picked up the most widely seen Cisco IPSec VPN configuration syntaxes, and tried to predict a solution with them. Currently most widely used IPSec VPN configuration is based on IKEv2, but I implemented IPSec VPN on lab routers with legacy IKEv1. As IKEv1 and IKEv2 are incompatible, the ChatGPT solution led to increasing problems.
The rest of the dialog with ChatGPT is provided below.
George >
I need your help to fix the issue with my network. I started getting complaints from users. They are unable to access our CRM from one of the branches. On the branch router, I see the following logs. Please provide the exact configuration to fix this issue.
*Jan 6 09:37:15.333: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down *Jan 6 09:37:15.338: %OSPF-5-ADJCHG: Process 1, Nbr 1.1.1.0 on Tunnel0 from FULL to DOWN, Neighbor Down: Interface down or detached *Jan 6 09:37:16.138: %CRYPTO-6-IKMP_MODE_FAILURE: Processing of Informational mode failed with peer at 10.0.0.1 *Jan 6 09:37:17.086: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=10.0.0.2, prot=50, spi=0x814B0503(2169177347), srcaddr=10.0.0.1, input interface=Tunnel0 *Jan 6 09:38:22.163: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for destaddr=10.0.0.2, prot=50, spi=0x814B0503(2169177347), srcaddr=10.0.0.1, input interface=Tunnel0 *Jan 6 09:39:02.155: %CRYPTO-6-IKMP_MODE_FAILURE: Processing of Informational mode failed with peer at 10.0.0.1
ChatGPT >
It looks like there may be an issue with the IPSec tunnel between your branch router and the peer at 10.0.0.1. The log messages indicate that the tunnel interface is down and that there are problems with the IPSec packets, specifically that the packets have an invalid SPI (Security Parameter Index). To troubleshoot this issue, you can try the following steps:
- Verify that the tunnel interface is up on both ends of the tunnel. You can do this by checking the status of the tunnel interface using the “show interface” command.
- Check the IPSec configuration on both ends of the tunnel to make sure that the IPSec policies match. This includes the transform sets, access lists, and any other configuration related to the IPSec tunnel.
- Check the status of the IPSec security associations (SAs) using the “show crypto isakmp sa” and “show crypto ipsec sa” commands. This will show you the current SAs for the tunnel and can help you identify any issues with the SAs.
- If the problem persists, you may need to clear the IPSec SAs and try to reestablish the tunnel. You can do this using the “clear crypto isakmp sa” and “clear crypto ipsec sa” commands.
I hope this helps! Let me know if you have any other questions.
George >
Hire is the “show interface” output. What else should I provide to you to determine the root cause of this issue?
R1#show interface
GigabitEthernet1 is up, line protocol is up
Hardware is CSR vNIC, address is 5254.0019.ab70 (bia 5254.0019.ab70)
Internet address is 10.0.0.2/30
MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full Duplex, 1000Mbps, link type is auto, media type is Virtual
output flow-control is unsupported, input flow-control is unsupported
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:09, output 00:00:09, output hang never
Last clearing of "show interface" counters never
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
406 packets input, 70200 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 0 pause input
407 packets output, 70716 bytes, 0 underruns
Output 0 broadcasts (0 IP multicasts)
0 output errors, 0 collisions, 2 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
1 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
Loopback0 is up, line protocol is up
Hardware is Loopback
Internet address is 1.1.1.1/32
MTU 1514 bytes, BW 8000000 Kbit/sec, DLY 5000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation LOOPBACK, loopback not set
Keepalive set (10 sec)
Last input 00:15:25, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
0 packets input, 0 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
56 packets output, 3960 bytes, 0 underruns
Output 0 broadcasts (0 IP multicasts)
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
Tunnel0 is up, line protocol is down
Hardware is Tunnel
Internet address is 11.0.0.2/30
MTU 10000 bytes, BW 100 Kbit/sec, DLY 50000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation TUNNEL, loopback not set
Keepalive not set
Tunnel linestate evaluation down - linestate protection reg down
Tunnel source 10.0.0.2, destination 10.0.0.1
Tunnel protocol/transport IPSEC/IP
Tunnel TTL 255
Tunnel transport MTU 1500 bytes
Tunnel transmit bandwidth 8000 (kbps)
Tunnel receive bandwidth 8000 (kbps)
Tunnel protection via IPSec (profile "AES256-SHA256")
Last input 00:06:05, output 00:06:06, output hang never
Last clearing of "show interface" counters 00:49:02
Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/0 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
115 packets input, 11144 bytes, 0 no buffer
Received 0 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
115 packets output, 10940 bytes, 0 underruns
Output 0 broadcasts (0 IP multicasts)
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 output buffer failures, 0 output buffers swapped out
<output was omitted for brevity>ChatGPT >
Thanks for providing the output of the “show interface” command. From the output, it looks like the Tunnel0 interface is not showing up. This could be the reason why the line protocol is down and why you are seeing the other log messages related to the IPSec tunnel.
To troubleshoot this issue further, it would be helpful to see the configuration of the Tunnel0 interface, as well as the IPSec configuration on the router. You can show these using the following commands:
show run interface Tunnel0
show run crypto
Additionally, it would be helpful to know more about your network topology and the devices that are involved in this IPSec tunnel. Can you provide more information about the devices and the network environment in which this tunnel is being used?
George >
Here is the output of commands you suggested to run:
R1#show run interface Tunnel0
Building configuration...
Current configuration : 186 bytes
!
interface Tunnel0 ip address 11.0.0.2 255.255.255.252
tunnel source 10.0.0.2
tunnel mode ipsec ipv4
tunnel destination 10.0.0.1
tunnel protection ipsec profile AES256-SHA256
end
R1#show run crypto
^
% Invalid input detected at '^' marker.
R1#
ChatGPT >
It looks like there may be a problem with the IPSec configuration on your router. The output of the “show run crypto” command indicates that there is an error in the configuration.
To troubleshoot this issue, you can try the following steps:
- Check the IPSec configuration for any syntax errors or missing parameters. You can do this by reviewing the output of the “show run crypto” command and looking for any issues.
- Verify that the IPSec policy that is being used by the Tunnel0 interface is properly configured. This includes the transform sets, access lists, and any other configuration related to the IPSec tunnel.
- Make sure that the IPSec peer (10.0.0.1 in this case) is reachable and that there are no connectivity issues between the two devices. You can do this by pinging the peer from the router or by checking the status of the tunnel interface on the peer device.
- If the problem persists, you may need to clear the IPSec SAs and try to reestablish the tunnel. You can do this using the “clear crypto isakmp sa” and “clear crypto ipsec sa” commands.
I hope this helps! Let me know if you have any other questions or if you need further assistance.
George >
It seems the router is not interpreting the “show run crypto” command. Please suggest a solution.
R1#show run crypto ^ % Invalid input detected at '^' marker. R1#
ChatGPT >
I apologize for the oversight. It looks like there is a problem with the “show run crypto” command on your router.
To troubleshoot this issue, you can try the following steps:
- Make sure that you are typing the command correctly and that there are no typos in the command.
- Check the version of the Cisco IOS software running on your router. Some versions of IOS do not support the “show run crypto” command. If this is the case, you can use the “show run” command to view the entire configuration of the router, and then search for the “crypto” keyword to find the relevant configuration.
- If the problem persists, there may be a problem with the Cisco IOS software or the device itself. In this case, you may need to reboot the router or upgrade the IOS software to a newer version.
I hope this helps! Let me know if you have any other questions or if you need further assistance.
George >
“show run” worked. Here is the entire configuration:
R1#show runn Building configuration... Current configuration : 4439 bytes ! ! Last configuration change at 09:36:38 UTC Fri Jan 6 2023 ! version 17.3 service timestamps debug datetime msec service timestamps log datetime msec ! Call-home is enabled by Smart-Licensing. service call-home platform qfp utilization monitor load 80 platform punt-keepalive disable-kernel-core platform console serial ! hostname R1 ! boot-start-marker boot-end-marker ! no aaa new-model ! ! login on-success log ! subscriber templating ! multilink bundle-name authenticated ! crypto pki trustpoint SLA-TrustPoint enrollment pkcs12 revocation-check crl ! ! crypto pki certificate chain SLA-TrustPoint certificate ca 01 30820321 30820209 A0030201 02020101 300D0609 2A864886 ! license udi pid CSR1000V sn XXXXXXXXX diagnostic bootup level minimal memory free low-watermark processor 71489 ! ! spanning-tree extend system-id ! ! redundancy ! crypto isakmp policy 1 encryption 3des hash sha512 authentication pre-share group 20 crypto isakmp key Cisco123 address 10.0.0.1 ! ! crypto ipsec transform-set AES256-SHA256 esp-aes 256 esp-sha256-hmac mode tunnel ! crypto ipsec profile AES256-SHA256 set transform-set AES256-SHA256 ! ! interface Loopback0 ip address 1.1.1.1 255.255.255.255 ! interface Tunnel0 ip address 11.0.0.2 255.255.255.252 tunnel source 10.0.0.2 tunnel mode ipsec ipv4 tunnel destination 10.0.0.1 tunnel protection ipsec profile AES256-SHA256 ! interface GigabitEthernet1 ip address 10.0.0.2 255.255.255.252 negotiation auto no mop enabled no mop sysid ! router ospf 1 network 11.0.0.0 0.0.0.255 area 1 ! ip forward-protocol nd no ip http server ip http secure-server ! ! control-plane ! . . . <output were omitted for brevity>
ChatGPT >
Thanks for providing the complete configuration of your router.
Based on the configuration, it looks like the Tunnel0 interface is being configured with the “tunnel mode ipsec ipv4” command, which indicates that this is an IPSec tunnel. The tunnel is being sourced from the interface with the IP address 10.0.0.2 and is being directed to the peer at 10.0.0.1.
The tunnel is also using an IPSec profile named “AES256-SHA256”, which specifies the transform sets and other parameters for the IPSec connection.
However, there is no explicit “crypto map” configuration in the router’s configuration. A crypto map is used to apply the IPSec policies to an interface, and it is necessary for establishing an IPSec tunnel.
To fix this issue, you will need to add a crypto map to your configuration and apply it to the Tunnel0 interface. Here is an example of how you can do this:
crypto map IPSEC-MAP 10 ipsec-isakmp set peer 10.0.0.1 set transform-set AES256-SHA256 set security-association lifetime seconds 3600 match address 100 interface Tunnel0 crypto map IPSEC-MAP
This configuration creates a crypto map named “IPSEC-MAP” and applies the transform set “AES256-SHA256” to it. The crypto map is then applied to the Tunnel0 interface, which will cause the router to initiate an IPSec tunnel to the peer at 10.0.0.1 using the specified transform set.
I hope this helps! Let me know if you have any other questions or if you need further assistance.
George>
I have implemented the configuration you suggested and received the following note:
R1(config-if)#crypto map IPSEC-MAP 10 ipsec-isakmp % NOTE: This new crypto map will remain disabled until a peer and a valid access list have been configured. R1(config-crypto-map)# set peer 10.0.0.1 R1(config-crypto-map)# set transform-set AES256-SHA256 R1(config-crypto-map)# set security-association lifetime seconds 3600 R1(config-crypto-map)# match address 100 R1(config-crypto-map)# R1(config-crypto-map)#interface Tunnel0 R1(config-if)# crypto map IPSEC-MAP % NOTE: crypto map is configured on tunnel or port-channel interface. Currently only GDOI crypto map is supported on tunnel or port-channel interface. R1(config-if)#
ChatGPT>
It looks like there is a problem with the crypto map configuration on your router. The error message “NOTE: This new crypto map will remain disabled until a peer and a valid access list have been configured” indicates that the crypto map is not complete and cannot be applied to the interface.
In addition, the error message “NOTE: crypto map is configured on tunnel or port-channel interface. Currently only GDOI crypto map is supported on tunnel or port-channel interface” indicates that the crypto map cannot be applied to the Tunnel0 interface because it is not supported on this type of interface.
To fix this issue, you will need to complete the configuration of the crypto map and make sure that it is applied to a supported interface. Here is an example of how you can do this:
access-list 100 permit ip any any crypto map IPSEC-MAP 10 ipsec-isakmp set peer 10.0.0.1 set transform-set AES256-SHA256 set security-association lifetime seconds 3600 match address 100 interface GigabitEthernet0/0 crypto map IPSEC-MAP
This configuration creates an access list named “100” that allows all traffic, and then applies it to the crypto map. The crypto map is then applied to the GigabitEthernet0/0 interface, which is a supported interface.
At this point, ChatGPT messed troubleshooting process. I see no reason to continue. This example demonstrates the stochastic nature of the ChatGPT algorithm.
ChatGPT can help to speed up finding problems in a network. It can derive practical high-level recommendations and structure a problem-finding approach. Usually, when you troubleshoot for hours, you think everything was checked and appropriately configured and start blaming software and hardware. ChatGPT can stand and provide recommendations and hints. But you should know how to ask questions, how AI works, and understand technology, at least 70% of it. To fully utilize its potential, it will be essential to develop skills in interacting with ChatGPT.
From the software developers’ point of view, ChatGPT is more robust, as it got more optimizations around software development. Nevertheless, it is a matter of time before ChatGPT can more accurately model Cisco IOS configuration and syntaxes.
Stay tuned!
