2025-12-01 04:25:50,812 - INFO - Domain Default found not creating 2025-12-01 04:25:51,022 - INFO - Project ctest-TestDropStats-22981093 not found, creating it 2025-12-01 04:25:51,636 - INFO - Created Project:ctest-TestDropStats-22981093, ID : eed38de3-3d15-4b55-9391-96e2d1d699ca 2025-12-01 04:25:53,414 - INFO - ================================================================================ 2025-12-01 04:25:53,415 - INFO - STARTING TEST : test_flow_action_drop_stats 2025-12-01 04:25:53,415 - INFO - TEST DESCRIPTION : Description: Verify flow action drop stats Steps: 1. Create 2 VNs and launch 1 VM in each VN, on different nodes on multi node setup 2. Add policy to deny pkts between VNs. 3. Ping VM2 from VM1 Pass criteria: 1. Verify flow action count gets incremented, get it from introspect 2. Verify flow action count gets incremented for the vmi of VM1, get it from introspect 2025-12-01 04:25:54,715 - DEBUG - Skipping xmpp flap check 2025-12-01 04:25:54,716 - INFO - Initial checks done. Running the testcase now 2025-12-01 04:25:54,716 - INFO - 2025-12-01 04:25:54,748 - DEBUG - Services list from nova: [, , , ] 2025-12-01 04:25:54,748 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-4827-1', 'an-jenkins-deploy-platform-ansible-os-4827-2']} 2025-12-01 04:25:55,517 - DEBUG - Response for create_network : {'network': {'id': '186ac307-bea1-451a-a54b-f516bba6fc18', 'name': 'ctest-vn-60352141', 'tenant_id': 'eed38de33d154b55939196e2d1d699ca', 'project_id': 'eed38de33d154b55939196e2d1d699ca', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDropStats-22981093', 'ctest-vn-60352141'], 'port_security_enabled': True, 'description': ''}} 2025-12-01 04:25:55,780 - DEBUG - Response for create_subnet : {'subnet': {'id': '501d36e5-9855-4f61-b323-736f99133ddd', 'name': '', 'tenant_id': 'eed38de33d154b55939196e2d1d699ca', 'network_id': '186ac307-bea1-451a-a54b-f516bba6fc18', 'ip_version': 4, 'cidr': '75.244.63.64/26', 'allocation_pools': [{'start': '75.244.63.66', 'end': '75.244.63.126'}], 'gateway_ip': '75.244.63.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '75.244.63.66', 'tags': [], 'project_id': 'eed38de33d154b55939196e2d1d699ca'}} 2025-12-01 04:25:55,804 - INFO - Created VN ctest-vn-60352141 2025-12-01 04:25:55,816 - DEBUG - VN ctest-vn-60352141 UUID is 186ac307-bea1-451a-a54b-f516bba6fc18 2025-12-01 04:25:56,231 - DEBUG - Response for create_network : {'network': {'id': 'f61a9835-6853-47c2-a7bf-3d5066b0e2dd', 'name': 'ctest-vn-18297694', 'tenant_id': 'eed38de33d154b55939196e2d1d699ca', 'project_id': 'eed38de33d154b55939196e2d1d699ca', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-TestDropStats-22981093', 'ctest-vn-18297694'], 'port_security_enabled': True, 'description': ''}} 2025-12-01 04:25:56,544 - DEBUG - Response for create_subnet : {'subnet': {'id': '90b32782-d888-44fc-a841-e59c29c6117a', 'name': '', 'tenant_id': 'eed38de33d154b55939196e2d1d699ca', 'network_id': 'f61a9835-6853-47c2-a7bf-3d5066b0e2dd', 'ip_version': 4, 'cidr': '51.219.84.128/26', 'allocation_pools': [{'start': '51.219.84.130', 'end': '51.219.84.190'}], 'gateway_ip': '51.219.84.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '51.219.84.130', 'tags': [], 'project_id': 'eed38de33d154b55939196e2d1d699ca'}} 2025-12-01 04:25:56,580 - INFO - Created VN ctest-vn-18297694 2025-12-01 04:25:56,598 - DEBUG - VN ctest-vn-18297694 UUID is f61a9835-6853-47c2-a7bf-3d5066b0e2dd 2025-12-01 04:25:57,975 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4827-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4827-1) 2025-12-01 04:26:00,001 - INFO - VM ([]) created on node: (an-jenkins-deploy-platform-ansible-os-4827-2), Zone: (nova:an-jenkins-deploy-platform-ansible-os-4827-2) 2025-12-01 04:26:00,002 - INFO - Waiting for VM ctest-vm1-37313625 to be up.. 2025-12-01 04:26:00,114 - DEBUG - VM is still in BUILD state, Expected: ACTIVE 2025-12-01 04:26:05,219 - DEBUG - VM is in ACTIVE state now 2025-12-01 04:26:05,219 - INFO - VM name : ctest-vm1-37313625 2025-12-01 04:26:05,315 - DEBUG - VM ctest-vm1-37313625 ID is a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:05,345 - DEBUG - VM ctest-vm1-37313625 launched on Node an-jenkins-deploy-platform-ansible-os-4827-1 2025-12-01 04:26:05,440 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:05,788 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:05,833 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/4362d7a1-3a2b-485a-87dd-387e010829c2 2025-12-01 04:26:09,028 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 0 received, 100% packet loss, time 1018ms') 2025-12-01 04:26:09,028 - DEBUG - Ping to Metadata IP 169.254.0.3 of VM ctest-vm1-37313625 failed! 2025-12-01 04:26:09,046 - DEBUG - Gateway for vn default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 is 75.244.63.65 and allocation pool is NOT set 2025-12-01 04:26:11,115 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=5.60 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=3.41 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 3.412/4.504/5.597/1.092 ms') 2025-12-01 04:26:11,115 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vm1-37313625 passed 2025-12-01 04:26:11,197 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-01 04:26:11,197 - DEBUG - Waiting to SSH to VM ctest-vm1-37313625, IP 75.244.63.67, Port 22 2025-12-01 04:26:11,269 - DEBUG - Error on ssh to cirros@169.254.0.3:22, result: /bin/bash: connect: Connection refused /bin/bash: line 1: /dev/tcp/169.254.0.3/22: Connection refused {'failed': True, 'command': '(echo > /dev/tcp/169.254.0.3/22)', 'real_command': '/bin/bash -l -c "(echo > /dev/tcp/169.254.0.3/22)"', 'return_code': 1, 'succeeded': False, 'stderr': ''} 2025-12-01 04:26:11,374 - DEBUG - VM ctest-vm1-37313625 is NOT ready for SSH connections, VM status: ACTIVE 2025-12-01 04:26:16,374 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-01 04:26:16,374 - DEBUG - Waiting to SSH to VM ctest-vm1-37313625, IP 75.244.63.67, Port 22 2025-12-01 04:26:16,548 - DEBUG - VM ctest-vm1-37313625 is ready for SSH connections 2025-12-01 04:26:16,548 - INFO - Waiting for VM ctest-vm2-62768666 to be up.. 2025-12-01 04:26:16,646 - DEBUG - VM is in ACTIVE state now 2025-12-01 04:26:16,646 - INFO - VM name : ctest-vm2-62768666 2025-12-01 04:26:16,742 - DEBUG - VM ctest-vm2-62768666 ID is 83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:16,742 - DEBUG - VM ctest-vm2-62768666 launched on Node an-jenkins-deploy-platform-ansible-os-4827-2 2025-12-01 04:26:16,836 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:16,849 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/5f6b39b0-21cb-4f52-9ee6-4a362fa926ce 2025-12-01 04:26:18,024 - DEBUG - (True, 'PING 169.254.0.3 (169.254.0.3) 56(84) bytes of data.\r\n64 bytes from 169.254.0.3: icmp_seq=1 ttl=63 time=3.12 ms\r\n64 bytes from 169.254.0.3: icmp_seq=2 ttl=63 time=1.04 ms\r\n\r\n--- 169.254.0.3 ping statistics ---\r\n2 packets transmitted, 2 received, 0% packet loss, time 1001ms\r\nrtt min/avg/max/mdev = 1.037/2.079/3.122/1.042 ms') 2025-12-01 04:26:18,024 - INFO - Ping to Metadata IP 169.254.0.3 of VM ctest-vm2-62768666 passed 2025-12-01 04:26:18,104 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-01 04:26:18,104 - DEBUG - Waiting to SSH to VM ctest-vm2-62768666, IP 51.219.84.131, Port 22 2025-12-01 04:26:18,275 - DEBUG - VM ctest-vm2-62768666 is ready for SSH connections 2025-12-01 04:26:18,373 - DEBUG - VM is in ACTIVE state now 2025-12-01 04:26:18,373 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-01 04:26:18,373 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:18,382 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/4362d7a1-3a2b-485a-87dd-387e010829c2 2025-12-01 04:26:18,393 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:18,402 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/4362d7a1-3a2b-485a-87dd-387e010829c2 2025-12-01 04:26:18,420 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/53278b9a-d5c3-487b-bae9-31f6868d71cf 2025-12-01 04:26:18,436 - DEBUG - Verifying in api server 10.0.0.23 2025-12-01 04:26:18,436 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:18,448 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/4362d7a1-3a2b-485a-87dd-387e010829c2 2025-12-01 04:26:18,461 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/53278b9a-d5c3-487b-bae9-31f6868d71cf 2025-12-01 04:26:18,473 - INFO - VM ctest-vm1-37313625 verfication in all API Servers passed 2025-12-01 04:26:18,496 - DEBUG - VM ctest-vm1-37313625 Tap interface: {'index': '3', 'name': 'tap4362d7a1-3a', 'uuid': '4362d7a1-3a2b-485a-87dd-387e010829c2', 'vrf_name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141:ctest-vn-60352141', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '25', 'l2_label': '29', 'vxlan_id': '5', 'vn_name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141', 'vm_uuid': 'a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb', 'vm_name': 'ctest-vm1-37313625', 'ip_addr': '75.244.63.67', 'mac_addr': '02:43:62:d7:a1:3a', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '166', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-22981093:4362d7a1-3a2b-485a-87dd-387e010829c2', 'sg_uuid_list': ['8c13bbd9-cd67-4e52-a5e3-09429648487b'], 'static_route_list': None, 'vm_project_uuid': 'eed38de3-3d15-4b55-9391-96e2d1d699ca', 'admin_state': 'Enabled', 'flow_key_idx': '28', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2025-12-01 04:26:18,496 - DEBUG - Agent 10.0.0.22 vrf name: default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141:ctest-vn-60352141 2025-12-01 04:26:18,504 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141:ctest-vn-60352141', 'ucindex': '2', 'mcindex': '2', 'l2index': '2', 'source': 'Config; ', 'uc6index': '2', 'vn': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141', 'table_label': '-1', 'vxlan_id': '5', 'evpnindex': '2', 'brindex': '2', 'mplsindex': '0', 'RD': '10.20.0.129:2', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2025-12-01 04:26:18,593 - DEBUG - VM ctest-vm1-37313625 : Tap interface tap4362d7a1-3a is set to Active 2025-12-01 04:26:18,593 - DEBUG - Tap interface tap4362d7a1-3a detail : {'index': '3', 'name': 'tap4362d7a1-3a', 'uuid': '4362d7a1-3a2b-485a-87dd-387e010829c2', 'vrf_name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141:ctest-vn-60352141', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '25', 'l2_label': '29', 'vxlan_id': '5', 'vn_name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141', 'vm_uuid': 'a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb', 'vm_name': 'ctest-vm1-37313625', 'ip_addr': '75.244.63.67', 'mac_addr': '02:43:62:d7:a1:3a', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '166', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-22981093:4362d7a1-3a2b-485a-87dd-387e010829c2', 'sg_uuid_list': ['8c13bbd9-cd67-4e52-a5e3-09429648487b'], 'static_route_list': None, 'vm_project_uuid': 'eed38de3-3d15-4b55-9391-96e2d1d699ca', 'admin_state': 'Enabled', 'flow_key_idx': '28', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2025-12-01 04:26:18,712 - DEBUG - Starting Layer 2 verification in Agent 2025-12-01 04:26:18,724 - DEBUG - Layer 2 path is seen for VM MAC 02:43:62:d7:a1:3a in agent 10.0.0.22 2025-12-01 04:26:18,724 - DEBUG - Active layer 2 route in agent is present for VMI tap4362d7a1-3a 2025-12-01 04:26:18,724 - DEBUG - L2 label(29) matches bw route table and interface table 2025-12-01 04:26:18,731 - DEBUG - VN default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 is found in Agent of node 10.0.0.22 2025-12-01 04:26:18,865 - DEBUG - VRF IDs of VN default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 is consistent in agent 10.0.0.22 2025-12-01 04:26:18,865 - DEBUG - Route for VM IP ['75.244.63.67'] is consistent in agent 10.0.0.22 2025-12-01 04:26:18,865 - DEBUG - VN default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 verification for VM ctest-vm1-37313625 in Agent 10.0.0.22 passed 2025-12-01 04:26:18,983 - DEBUG - Starting all layer 2 verification in agent 10.0.0.22 2025-12-01 04:26:18,994 - DEBUG - Route for VM MAC 02:43:62:d7:a1:3a is consistent in agent 10.0.0.22 2025-12-01 04:26:19,000 - INFO - VM ctest-vm1-37313625 verifications in Compute nodes passed 2025-12-01 04:26:19,257 - DEBUG - Validated VM route 75.244.63.67 in vrouter of 10.0.0.22 2025-12-01 04:26:19,262 - DEBUG - Remote Nh validation passed. Expected Tunnel, label: None, 0; Got None,0 2025-12-01 04:26:19,263 - INFO - Validated routes of VM ctest-vm1-37313625 in all vrouters 2025-12-01 04:26:19,845 - DEBUG - Starting all layer2 verification in 10.0.0.27 Control Node 2025-12-01 04:26:19,985 - DEBUG - Layer2 route found for VM MAC 02:43:62:d7:a1:3a in Control-node 10.0.0.27 2025-12-01 04:26:19,985 - DEBUG - L2 Label for VM ctest-vm1-37313625 same between Control-node 10.0.0.27 and Agent, Expected: 29, Seen: 29 2025-12-01 04:26:19,985 - DEBUG - Starting all layer2 verification in 10.0.0.33 Control Node 2025-12-01 04:26:20,111 - DEBUG - Layer2 route found for VM MAC 02:43:62:d7:a1:3a in Control-node 10.0.0.33 2025-12-01 04:26:20,111 - DEBUG - L2 Label for VM ctest-vm1-37313625 same between Control-node 10.0.0.33 and Agent, Expected: 29, Seen: 29 2025-12-01 04:26:20,111 - INFO - Verification in Control-nodes for VM ctest-vm1-37313625 passed 2025-12-01 04:26:20,111 - DEBUG - Verifying the vm in opserver 2025-12-01 04:26:20,111 - DEBUG - Verifying in collector 10.0.0.23 ... 2025-12-01 04:26:20,111 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb?flat 2025-12-01 04:26:20,457 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb?flat 2025-12-01 04:26:20,535 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-22981093:4362d7a1-3a2b-485a-87dd-387e010829c2?flat 2025-12-01 04:26:20,883 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-22981093:4362d7a1-3a2b-485a-87dd-387e010829c2?flat 2025-12-01 04:26:20,979 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-22981093:4362d7a1-3a2b-485a-87dd-387e010829c2?flat 2025-12-01 04:26:21,058 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb?flat 2025-12-01 04:26:21,126 - DEBUG - Verifying vm in vn uve 2025-12-01 04:26:21,126 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-22981093:4362d7a1-3a2b-485a-87dd-387e010829c2?flat 2025-12-01 04:26:21,218 - DEBUG - VM uve shows interface as {'fixed_ip4_list': ['75.244.63.67'], 'mac_address': '02:43:62:d7:a1:3a', 'vn_uuid': '186ac307-bea1-451a-a54b-f516bba6fc18', 'gateway': '75.244.63.65', 'vhostuser_mode': 0, 'label': 25, 'ip6_address': '::', 'tx_vlan': 65535, 'virtual_network': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141', 'active': True, 'port_mirror_enabled': False, 'ip_address': '75.244.63.67', '__T': 1764563175182369, 'uuid': '4362d7a1-3a2b-485a-87dd-387e010829c2', 'vm_uuid': 'a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb', 'admin_state': True, 'l2_active': True, 'rx_vlan': 65535, 'vm_name': 'ctest-vm1-37313625', 'ip4_active': True, 'ip6_active': False, 'is_health_check_active': True} 2025-12-01 04:26:21,218 - DEBUG - VM uve shows ip address as ['75.244.63.67', '::'] 2025-12-01 04:26:21,218 - DEBUG - VM uve shows virtual network as default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 2025-12-01 04:26:21,218 - INFO - Verifying through opserver in 10.0.0.27 2025-12-01 04:26:21,218 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141?flat 2025-12-01 04:26:21,243 - DEBUG - expected vm list a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:21,243 - DEBUG - Extracted vm list ['a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb'] 2025-12-01 04:26:21,243 - INFO - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is present in default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 2025-12-01 04:26:21,243 - DEBUG - Verifying vm in vrouter uve 2025-12-01 04:26:21,243 - DEBUG - Getting info from collector 10.0.0.23.. 2025-12-01 04:26:21,243 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb?flat 2025-12-01 04:26:21,311 - DEBUG - VM uve shows vrouter as ['an-jenkins-deploy-platform-ansible-os-4827-1'] 2025-12-01 04:26:21,311 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:26:21,369 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:26:21,722 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:26:21,811 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:26:21,812 - DEBUG - Validated that VM ctest-vm1-37313625 is in Vrouter an-jenkins-deploy-platform-ansible-os-4827-1 UVE 2025-12-01 04:26:21,812 - DEBUG - Verifying vm tap interface/vn in vrouter uve 2025-12-01 04:26:21,819 - DEBUG - Expected tap interface of VM uuid a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is default-domain:ctest-TestDropStats-22981093:4362d7a1-3a2b-485a-87dd-387e010829c2 2025-12-01 04:26:21,819 - DEBUG - Expected VN of VM uuid a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 2025-12-01 04:26:21,819 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:26:21,869 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:26:21,955 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:26:21,955 - INFO - tap interface default-domain:ctest-TestDropStats-22981093:4362d7a1-3a2b-485a-87dd-387e010829c2 of vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:26:21,955 - DEBUG - Connected network default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 of vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:26:21,955 - DEBUG - Verifying the a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb virtual network link through opserver 10.0.0.27 2025-12-01 04:26:21,955 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machines 2025-12-01 04:26:21,968 - DEBUG - VM link and name as {'name': 'a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb?flat'} 2025-12-01 04:26:21,968 - DEBUG - Verifying the a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb virtual network link through opserver 10.0.0.33 2025-12-01 04:26:21,968 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-12-01 04:26:22,037 - DEBUG - VM link and name as {'name': 'a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb', 'href': 'http://10.0.0.33:8081/analytics/uves/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb?flat'} 2025-12-01 04:26:22,037 - DEBUG - Verifying the a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb virtual network link through opserver 10.0.0.23 2025-12-01 04:26:22,037 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2025-12-01 04:26:22,051 - DEBUG - VM link and name as {'name': 'a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb?flat'} 2025-12-01 04:26:22,051 - INFO - VM ctest-vm1-37313625 validations in Opserver passed 2025-12-01 04:26:22,147 - DEBUG - VM is in ACTIVE state now 2025-12-01 04:26:22,147 - DEBUG - verify_vm_launched is skipped since self.vm_launch_flag is set 2025-12-01 04:26:22,147 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:22,158 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/5f6b39b0-21cb-4f52-9ee6-4a362fa926ce 2025-12-01 04:26:22,172 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:22,182 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/5f6b39b0-21cb-4f52-9ee6-4a362fa926ce 2025-12-01 04:26:22,195 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/3233bfc5-fa5a-410a-bb29-5144f2f3e984 2025-12-01 04:26:22,207 - DEBUG - Verifying in api server 10.0.0.23 2025-12-01 04:26:22,207 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:22,216 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine-interface/5f6b39b0-21cb-4f52-9ee6-4a362fa926ce 2025-12-01 04:26:22,228 - DEBUG - Requesting: http://10.0.0.23:8082/instance-ip/3233bfc5-fa5a-410a-bb29-5144f2f3e984 2025-12-01 04:26:22,240 - INFO - VM ctest-vm2-62768666 verfication in all API Servers passed 2025-12-01 04:26:22,259 - DEBUG - VM ctest-vm2-62768666 Tap interface: {'index': '3', 'name': 'tap5f6b39b0-21', 'uuid': '5f6b39b0-21cb-4f52-9ee6-4a362fa926ce', 'vrf_name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694:ctest-vn-18297694', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '25', 'l2_label': '29', 'vxlan_id': '7', 'vn_name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694', 'vm_uuid': '83109cc2-bdc4-438e-9eba-07475b31157e', 'vm_name': 'ctest-vm2-62768666', 'ip_addr': '51.219.84.131', 'mac_addr': '02:5f:6b:39:b0:21', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '136', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-22981093:5f6b39b0-21cb-4f52-9ee6-4a362fa926ce', 'sg_uuid_list': ['8c13bbd9-cd67-4e52-a5e3-09429648487b'], 'static_route_list': None, 'vm_project_uuid': 'eed38de3-3d15-4b55-9391-96e2d1d699ca', 'admin_state': 'Enabled', 'flow_key_idx': '28', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2025-12-01 04:26:22,259 - DEBUG - Agent 10.0.0.241 vrf name: default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694:ctest-vn-18297694 2025-12-01 04:26:22,266 - DEBUG - Agent VRF Object : {'vrf_list': [{'name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694:ctest-vn-18297694', 'ucindex': '2', 'mcindex': '2', 'l2index': '2', 'source': 'Config; ', 'uc6index': '2', 'vn': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694', 'table_label': '-1', 'vxlan_id': '7', 'evpnindex': '2', 'brindex': '2', 'mplsindex': '0', 'RD': '10.20.0.14:2', 'mac_aging_time': '0', 'layer2_control_word': 'false', 'forwarding_vrf': None, 'hbf_rintf': '-1', 'hbf_lintf': '-1'}]} 2025-12-01 04:26:22,348 - DEBUG - VM ctest-vm2-62768666 : Tap interface tap5f6b39b0-21 is set to Active 2025-12-01 04:26:22,349 - DEBUG - Tap interface tap5f6b39b0-21 detail : {'index': '3', 'name': 'tap5f6b39b0-21', 'uuid': '5f6b39b0-21cb-4f52-9ee6-4a362fa926ce', 'vrf_name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694:ctest-vn-18297694', 'active': 'Active', 'ipv4_active': 'Active', 'l2_active': 'L2 Active', 'ip6_active': 'Ipv6 Inactive < no-ipv6-addr >', 'health_check_active': 'Active', 'dhcp_service': 'Enable', 'dhcp_service_v6': 'Disable', 'dns_service': 'Enable', 'type': 'vport', 'label': '25', 'l2_label': '29', 'vxlan_id': '7', 'vn_name': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694', 'vm_uuid': '83109cc2-bdc4-438e-9eba-07475b31157e', 'vm_name': 'ctest-vm2-62768666', 'ip_addr': '51.219.84.131', 'mac_addr': '02:5f:6b:39:b0:21', 'policy': 'Enable', 'fip_list': [], 'mdata_ip_addr': '169.254.0.3', 'service_vlan_list': None, 'os_ifindex': '136', 'fabric_port': 'NotFabricPort', 'alloc_linklocal_ip': 'LL-Enable', 'analyzer_name': None, 'config_name': 'default-domain:ctest-TestDropStats-22981093:5f6b39b0-21cb-4f52-9ee6-4a362fa926ce', 'sg_uuid_list': ['8c13bbd9-cd67-4e52-a5e3-09429648487b'], 'static_route_list': None, 'vm_project_uuid': 'eed38de3-3d15-4b55-9391-96e2d1d699ca', 'admin_state': 'Enabled', 'flow_key_idx': '28', 'allowed_address_pair_list': None, 'ip6_addr': '::', 'local_preference': '0', 'tx_vlan_id': '-1', 'rx_vlan_id': '-1', 'parent_interface': None, 'subnet': '--NA--', 'sub_type': 'Tap', 'vrf_assign_acl_uuid': '--NA--', 'vmi_type': 'Virtual Machine', 'transport': 'Ethernet', 'logical_interface_uuid': '00000000-0000-0000-0000-000000000000', 'flood_unknown_unicast': 'false', 'physical_device': None, 'physical_interface': None, 'fixed_ip4_list': None, 'fixed_ip6_list': None, 'fat_flow_list': None, 'metadata_ip_active': 'Active', 'service_health_check_ip': '0.0.0.0', 'alias_ip_list': None, 'drop_new_flows': 'false', 'bridge_domain_list': [], 'vmi_tag_list': None, 'policy_set_acl_list': None, 'slo_list': None, 'vhostuser_mode': '0', 'si_other_end_vmi': '00000000-0000-0000-0000-000000000000', 'cfg_igmp_enable': 'false', 'igmp_enabled': 'false', 'max_flows': '0', 'policy_set_fwaas_list': None, 'bond_interface_list': None, 'mac_ip_learning_enable': 'false', 'mac_ip_list': []} 2025-12-01 04:26:22,467 - DEBUG - Starting Layer 2 verification in Agent 2025-12-01 04:26:22,478 - DEBUG - Layer 2 path is seen for VM MAC 02:5f:6b:39:b0:21 in agent 10.0.0.241 2025-12-01 04:26:22,478 - DEBUG - Active layer 2 route in agent is present for VMI tap5f6b39b0-21 2025-12-01 04:26:22,478 - DEBUG - L2 label(29) matches bw route table and interface table 2025-12-01 04:26:22,491 - DEBUG - VN default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 is found in Agent of node 10.0.0.241 2025-12-01 04:26:22,632 - DEBUG - VRF IDs of VN default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 is consistent in agent 10.0.0.241 2025-12-01 04:26:22,632 - DEBUG - Route for VM IP ['51.219.84.131'] is consistent in agent 10.0.0.241 2025-12-01 04:26:22,632 - DEBUG - VN default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 verification for VM ctest-vm2-62768666 in Agent 10.0.0.241 passed 2025-12-01 04:26:22,749 - DEBUG - Starting all layer 2 verification in agent 10.0.0.241 2025-12-01 04:26:22,762 - DEBUG - Route for VM MAC 02:5f:6b:39:b0:21 is consistent in agent 10.0.0.241 2025-12-01 04:26:22,762 - INFO - VM ctest-vm2-62768666 verifications in Compute nodes passed 2025-12-01 04:26:23,033 - DEBUG - Validated VM route 51.219.84.131 in vrouter of 10.0.0.241 2025-12-01 04:26:23,039 - DEBUG - Remote Nh validation passed. Expected Tunnel, label: None, 0; Got None,0 2025-12-01 04:26:23,039 - INFO - Validated routes of VM ctest-vm2-62768666 in all vrouters 2025-12-01 04:26:23,601 - DEBUG - Starting all layer2 verification in 10.0.0.27 Control Node 2025-12-01 04:26:23,729 - DEBUG - Layer2 route found for VM MAC 02:5f:6b:39:b0:21 in Control-node 10.0.0.27 2025-12-01 04:26:23,730 - DEBUG - L2 Label for VM ctest-vm2-62768666 same between Control-node 10.0.0.27 and Agent, Expected: 29, Seen: 29 2025-12-01 04:26:23,730 - DEBUG - Starting all layer2 verification in 10.0.0.33 Control Node 2025-12-01 04:26:23,860 - DEBUG - Layer2 route found for VM MAC 02:5f:6b:39:b0:21 in Control-node 10.0.0.33 2025-12-01 04:26:23,860 - DEBUG - L2 Label for VM ctest-vm2-62768666 same between Control-node 10.0.0.33 and Agent, Expected: 29, Seen: 29 2025-12-01 04:26:23,860 - INFO - Verification in Control-nodes for VM ctest-vm2-62768666 passed 2025-12-01 04:26:23,861 - DEBUG - Verifying the vm in opserver 2025-12-01 04:26:23,861 - DEBUG - Verifying in collector 10.0.0.23 ... 2025-12-01 04:26:23,861 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e?flat 2025-12-01 04:26:23,882 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-22981093:5f6b39b0-21cb-4f52-9ee6-4a362fa926ce?flat 2025-12-01 04:26:23,903 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-22981093:5f6b39b0-21cb-4f52-9ee6-4a362fa926ce?flat 2025-12-01 04:26:23,923 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e?flat 2025-12-01 04:26:23,941 - DEBUG - Verifying vm in vn uve 2025-12-01 04:26:23,942 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machine-interface/default-domain:ctest-TestDropStats-22981093:5f6b39b0-21cb-4f52-9ee6-4a362fa926ce?flat 2025-12-01 04:26:23,966 - DEBUG - VM uve shows interface as {'vn_uuid': 'f61a9835-6853-47c2-a7bf-3d5066b0e2dd', 'ip_address': '51.219.84.131', 'fixed_ip4_list': ['51.219.84.131'], 'label': 25, 'vhostuser_mode': 0, 'is_health_check_active': True, 'ip6_active': False, 'l2_active': True, 'active': True, 'vm_name': 'ctest-vm2-62768666', 'virtual_network': 'default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694', 'mac_address': '02:5f:6b:39:b0:21', 'tx_vlan': 65535, 'admin_state': True, 'port_mirror_enabled': False, 'uuid': '5f6b39b0-21cb-4f52-9ee6-4a362fa926ce', 'gateway': '51.219.84.129', 'vm_uuid': '83109cc2-bdc4-438e-9eba-07475b31157e', 'rx_vlan': 65535, 'ip4_active': True, '__T': 1764563183759908, 'ip6_address': '::'} 2025-12-01 04:26:23,966 - DEBUG - VM uve shows ip address as ['51.219.84.131', '::'] 2025-12-01 04:26:23,966 - DEBUG - VM uve shows virtual network as default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 2025-12-01 04:26:23,966 - INFO - Verifying through opserver in 10.0.0.27 2025-12-01 04:26:23,966 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694?flat 2025-12-01 04:26:23,991 - DEBUG - expected vm list 83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:23,991 - DEBUG - Extracted vm list ['83109cc2-bdc4-438e-9eba-07475b31157e'] 2025-12-01 04:26:23,991 - INFO - VM 83109cc2-bdc4-438e-9eba-07475b31157e is present in default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 2025-12-01 04:26:23,991 - DEBUG - Verifying vm in vrouter uve 2025-12-01 04:26:23,991 - DEBUG - Getting info from collector 10.0.0.23.. 2025-12-01 04:26:23,991 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e?flat 2025-12-01 04:26:24,011 - DEBUG - VM uve shows vrouter as ['an-jenkins-deploy-platform-ansible-os-4827-2'] 2025-12-01 04:26:24,011 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-2:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:26:24,069 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-2?flat 2025-12-01 04:26:24,100 - DEBUG - VM 83109cc2-bdc4-438e-9eba-07475b31157e is present in vrouter an-jenkins-deploy-platform-ansible-os-4827-2 uve 2025-12-01 04:26:24,100 - DEBUG - Validated that VM ctest-vm2-62768666 is in Vrouter an-jenkins-deploy-platform-ansible-os-4827-2 UVE 2025-12-01 04:26:24,100 - DEBUG - Verifying vm tap interface/vn in vrouter uve 2025-12-01 04:26:24,108 - DEBUG - Expected tap interface of VM uuid 83109cc2-bdc4-438e-9eba-07475b31157e is default-domain:ctest-TestDropStats-22981093:5f6b39b0-21cb-4f52-9ee6-4a362fa926ce 2025-12-01 04:26:24,108 - DEBUG - Expected VN of VM uuid 83109cc2-bdc4-438e-9eba-07475b31157e is default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 2025-12-01 04:26:24,108 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-2:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:26:24,157 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-2?flat 2025-12-01 04:26:24,191 - DEBUG - VM 83109cc2-bdc4-438e-9eba-07475b31157e is present in vrouter an-jenkins-deploy-platform-ansible-os-4827-2 uve 2025-12-01 04:26:24,191 - INFO - tap interface default-domain:ctest-TestDropStats-22981093:5f6b39b0-21cb-4f52-9ee6-4a362fa926ce of vm 83109cc2-bdc4-438e-9eba-07475b31157e is present in vrouter an-jenkins-deploy-platform-ansible-os-4827-2 uve 2025-12-01 04:26:24,191 - DEBUG - Connected network default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 of vm 83109cc2-bdc4-438e-9eba-07475b31157e is present in vrouter an-jenkins-deploy-platform-ansible-os-4827-2 uve 2025-12-01 04:26:24,192 - DEBUG - Verifying the 83109cc2-bdc4-438e-9eba-07475b31157e virtual network link through opserver 10.0.0.27 2025-12-01 04:26:24,192 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machines 2025-12-01 04:26:24,205 - DEBUG - VM link and name as {'name': '83109cc2-bdc4-438e-9eba-07475b31157e', 'href': 'http://10.0.0.27:8081/analytics/uves/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e?flat'} 2025-12-01 04:26:24,205 - DEBUG - Verifying the 83109cc2-bdc4-438e-9eba-07475b31157e virtual network link through opserver 10.0.0.33 2025-12-01 04:26:24,205 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-12-01 04:26:24,269 - DEBUG - VM link and name as {'name': '83109cc2-bdc4-438e-9eba-07475b31157e', 'href': 'http://10.0.0.33:8081/analytics/uves/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e?flat'} 2025-12-01 04:26:24,269 - DEBUG - Verifying the 83109cc2-bdc4-438e-9eba-07475b31157e virtual network link through opserver 10.0.0.23 2025-12-01 04:26:24,269 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2025-12-01 04:26:24,284 - DEBUG - VM link and name as {'name': '83109cc2-bdc4-438e-9eba-07475b31157e', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e?flat'} 2025-12-01 04:26:24,284 - INFO - VM ctest-vm2-62768666 validations in Opserver passed 2025-12-01 04:26:24,306 - INFO - ---------------------------------------------------------------------------------------------------- 2025-12-01 04:26:24,306 - INFO - ds_discard : 3 2025-12-01 04:26:24,306 - INFO - ds_pull : 0 2025-12-01 04:26:24,306 - INFO - ds_invalid_if : 0 2025-12-01 04:26:24,306 - INFO - ds_invalid_arp : 0 2025-12-01 04:26:24,306 - INFO - ds_trap_no_if : 0 2025-12-01 04:26:24,306 - INFO - ds_nowhere_to_go : 0 2025-12-01 04:26:24,306 - INFO - ds_flow_queue_limit_exceeded : 0 2025-12-01 04:26:24,306 - INFO - ds_flow_no_memory : 0 2025-12-01 04:26:24,306 - INFO - ds_flow_invalid_protocol : 0 2025-12-01 04:26:24,307 - INFO - ds_flow_nat_no_rflow : 0 2025-12-01 04:26:24,307 - INFO - ds_flow_action_drop : 0 2025-12-01 04:26:24,307 - INFO - ds_flow_action_invalid : 0 2025-12-01 04:26:24,307 - INFO - ds_flow_unusable : 0 2025-12-01 04:26:24,307 - INFO - ds_flow_table_full : 0 2025-12-01 04:26:24,307 - INFO - ds_interface_tx_discard : 0 2025-12-01 04:26:24,307 - INFO - ds_interface_drop : 0 2025-12-01 04:26:24,307 - INFO - ds_duplicated : 0 2025-12-01 04:26:24,307 - INFO - ds_push : 0 2025-12-01 04:26:24,307 - INFO - ds_ttl_exceeded : 0 2025-12-01 04:26:24,307 - INFO - ds_invalid_nh : 0 2025-12-01 04:26:24,307 - INFO - ds_invalid_label : 0 2025-12-01 04:26:24,307 - INFO - ds_invalid_protocol : 0 2025-12-01 04:26:24,307 - INFO - ds_interface_rx_discard : 0 2025-12-01 04:26:24,307 - INFO - ds_invalid_mcast_source : 0 2025-12-01 04:26:24,307 - INFO - ds_head_alloc_fail : 0 2025-12-01 04:26:24,307 - INFO - ds_pcow_fail : 0 2025-12-01 04:26:24,307 - INFO - ds_mcast_clone_fail : 0 2025-12-01 04:26:24,307 - INFO - ds_rewrite_fail : 0 2025-12-01 04:26:24,307 - INFO - ds_misc : 0 2025-12-01 04:26:24,307 - INFO - ds_invalid_packet : 0 2025-12-01 04:26:24,307 - INFO - ds_cksum_err : 0 2025-12-01 04:26:24,307 - INFO - ds_no_fmd : 0 2025-12-01 04:26:24,308 - INFO - ds_invalid_vnid : 0 2025-12-01 04:26:24,308 - INFO - ds_frag_err : 0 2025-12-01 04:26:24,308 - INFO - ds_invalid_source : 0 2025-12-01 04:26:24,308 - INFO - ds_mcast_df_bit : 0 2025-12-01 04:26:24,308 - INFO - ds_l2_no_route : 0 2025-12-01 04:26:24,308 - INFO - ds_vlan_fwd_tx : 0 2025-12-01 04:26:24,308 - INFO - ds_vlan_fwd_enq : 0 2025-12-01 04:26:24,308 - INFO - ds_drop_new_flow : 0 2025-12-01 04:26:24,308 - INFO - ds_flow_evict : 0 2025-12-01 04:26:24,308 - INFO - ds_trap_original : 0 2025-12-01 04:26:24,308 - INFO - ds_no_memory : 0 2025-12-01 04:26:24,308 - INFO - ds_fragment_queue_fail : 0 2025-12-01 04:26:24,308 - INFO - ds_no_frag_entry : 0 2025-12-01 04:26:24,308 - INFO - ds_icmp_error : 0 2025-12-01 04:26:24,308 - INFO - ds_clone_fail : 0 2025-12-01 04:26:24,308 - INFO - ds_drop_pkts : 3 2025-12-01 04:26:24,315 - INFO - ---------------------------------------------------------------------------------------------------- 2025-12-01 04:26:24,315 - INFO - ds_discard : 6 2025-12-01 04:26:24,315 - INFO - ds_pull : 0 2025-12-01 04:26:24,315 - INFO - ds_invalid_if : 0 2025-12-01 04:26:24,315 - INFO - ds_invalid_arp : 0 2025-12-01 04:26:24,315 - INFO - ds_trap_no_if : 0 2025-12-01 04:26:24,316 - INFO - ds_nowhere_to_go : 0 2025-12-01 04:26:24,316 - INFO - ds_flow_queue_limit_exceeded : 0 2025-12-01 04:26:24,316 - INFO - ds_flow_no_memory : 0 2025-12-01 04:26:24,316 - INFO - ds_flow_invalid_protocol : 0 2025-12-01 04:26:24,316 - INFO - ds_flow_nat_no_rflow : 0 2025-12-01 04:26:24,316 - INFO - ds_flow_action_drop : 16 2025-12-01 04:26:24,316 - INFO - ds_flow_action_invalid : 0 2025-12-01 04:26:24,316 - INFO - ds_flow_unusable : 0 2025-12-01 04:26:24,316 - INFO - ds_flow_table_full : 0 2025-12-01 04:26:24,316 - INFO - ds_interface_tx_discard : 0 2025-12-01 04:26:24,316 - INFO - ds_interface_drop : 0 2025-12-01 04:26:24,316 - INFO - ds_duplicated : 0 2025-12-01 04:26:24,316 - INFO - ds_push : 0 2025-12-01 04:26:24,316 - INFO - ds_ttl_exceeded : 0 2025-12-01 04:26:24,316 - INFO - ds_invalid_nh : 5 2025-12-01 04:26:24,316 - INFO - ds_invalid_label : 0 2025-12-01 04:26:24,316 - INFO - ds_invalid_protocol : 0 2025-12-01 04:26:24,316 - INFO - ds_interface_rx_discard : 0 2025-12-01 04:26:24,316 - INFO - ds_invalid_mcast_source : 0 2025-12-01 04:26:24,316 - INFO - ds_head_alloc_fail : 0 2025-12-01 04:26:24,316 - INFO - ds_pcow_fail : 0 2025-12-01 04:26:24,316 - INFO - ds_mcast_clone_fail : 0 2025-12-01 04:26:24,316 - INFO - ds_rewrite_fail : 0 2025-12-01 04:26:24,316 - INFO - ds_misc : 0 2025-12-01 04:26:24,317 - INFO - ds_invalid_packet : 0 2025-12-01 04:26:24,317 - INFO - ds_cksum_err : 0 2025-12-01 04:26:24,317 - INFO - ds_no_fmd : 0 2025-12-01 04:26:24,317 - INFO - ds_invalid_vnid : 0 2025-12-01 04:26:24,317 - INFO - ds_frag_err : 0 2025-12-01 04:26:24,317 - INFO - ds_invalid_source : 0 2025-12-01 04:26:24,317 - INFO - ds_mcast_df_bit : 0 2025-12-01 04:26:24,317 - INFO - ds_l2_no_route : 2 2025-12-01 04:26:24,317 - INFO - ds_vlan_fwd_tx : 0 2025-12-01 04:26:24,317 - INFO - ds_vlan_fwd_enq : 0 2025-12-01 04:26:24,317 - INFO - ds_drop_new_flow : 0 2025-12-01 04:26:24,317 - INFO - ds_flow_evict : 0 2025-12-01 04:26:24,317 - INFO - ds_trap_original : 0 2025-12-01 04:26:24,317 - INFO - ds_no_memory : 0 2025-12-01 04:26:24,317 - INFO - ds_fragment_queue_fail : 0 2025-12-01 04:26:24,317 - INFO - ds_no_frag_entry : 0 2025-12-01 04:26:24,317 - INFO - ds_icmp_error : 0 2025-12-01 04:26:24,317 - INFO - ds_clone_fail : 0 2025-12-01 04:26:24,317 - INFO - ds_drop_pkts : 29 2025-12-01 04:26:24,317 - DEBUG - Running remote_cmd, Cmd : ping -s 56 -c 5 -W 1 51.219.84.131, host_string: cirros@169.254.0.3, password: cubswin:)gateway: ubuntu@10.0.0.22, gateway password: c0ntrail123 2025-12-01 04:26:24,318 - DEBUG - ping -s 56 -c 5 -W 1 51.219.84.131 2025-12-01 04:26:30,370 - DEBUG - PING 51.219.84.131 (51.219.84.131): 56 data bytes --- 51.219.84.131 ping statistics --- 5 packets transmitted, 0 packets received, 100% packet loss 2025-12-01 04:26:30,370 - WARNING - Ping to IP 51.219.84.131 from VM ctest-vm1-37313625 failed 2025-12-01 04:26:46,532 - INFO - Vif and vrouter dropstats of type ds_flow_action_drop verifed 2025-12-01 04:26:46,532 - INFO - ---------------------------------------------------------------------------------------------------- 2025-12-01 04:26:46,532 - INFO - ds_discard : 3 2025-12-01 04:26:46,532 - INFO - ds_pull : 0 2025-12-01 04:26:46,532 - INFO - ds_invalid_if : 0 2025-12-01 04:26:46,532 - INFO - ds_invalid_arp : 0 2025-12-01 04:26:46,532 - INFO - ds_trap_no_if : 0 2025-12-01 04:26:46,532 - INFO - ds_nowhere_to_go : 0 2025-12-01 04:26:46,532 - INFO - ds_flow_queue_limit_exceeded : 0 2025-12-01 04:26:46,532 - INFO - ds_flow_no_memory : 0 2025-12-01 04:26:46,532 - INFO - ds_flow_invalid_protocol : 0 2025-12-01 04:26:46,532 - INFO - ds_flow_nat_no_rflow : 0 2025-12-01 04:26:46,532 - INFO - ds_flow_action_drop : 5 2025-12-01 04:26:46,532 - INFO - ds_flow_action_invalid : 0 2025-12-01 04:26:46,532 - INFO - ds_flow_unusable : 0 2025-12-01 04:26:46,532 - INFO - ds_flow_table_full : 0 2025-12-01 04:26:46,532 - INFO - ds_interface_tx_discard : 0 2025-12-01 04:26:46,533 - INFO - ds_interface_drop : 0 2025-12-01 04:26:46,533 - INFO - ds_duplicated : 0 2025-12-01 04:26:46,533 - INFO - ds_push : 0 2025-12-01 04:26:46,533 - INFO - ds_ttl_exceeded : 0 2025-12-01 04:26:46,533 - INFO - ds_invalid_nh : 0 2025-12-01 04:26:46,533 - INFO - ds_invalid_label : 0 2025-12-01 04:26:46,533 - INFO - ds_invalid_protocol : 0 2025-12-01 04:26:46,533 - INFO - ds_interface_rx_discard : 0 2025-12-01 04:26:46,533 - INFO - ds_invalid_mcast_source : 0 2025-12-01 04:26:46,533 - INFO - ds_head_alloc_fail : 0 2025-12-01 04:26:46,533 - INFO - ds_pcow_fail : 0 2025-12-01 04:26:46,533 - INFO - ds_mcast_clone_fail : 0 2025-12-01 04:26:46,533 - INFO - ds_rewrite_fail : 0 2025-12-01 04:26:46,533 - INFO - ds_misc : 0 2025-12-01 04:26:46,533 - INFO - ds_invalid_packet : 0 2025-12-01 04:26:46,533 - INFO - ds_cksum_err : 0 2025-12-01 04:26:46,533 - INFO - ds_no_fmd : 0 2025-12-01 04:26:46,533 - INFO - ds_invalid_vnid : 0 2025-12-01 04:26:46,533 - INFO - ds_frag_err : 0 2025-12-01 04:26:46,533 - INFO - ds_invalid_source : 0 2025-12-01 04:26:46,533 - INFO - ds_mcast_df_bit : 0 2025-12-01 04:26:46,533 - INFO - ds_l2_no_route : 0 2025-12-01 04:26:46,533 - INFO - ds_vlan_fwd_tx : 0 2025-12-01 04:26:46,534 - INFO - ds_vlan_fwd_enq : 0 2025-12-01 04:26:46,534 - INFO - ds_drop_new_flow : 0 2025-12-01 04:26:46,534 - INFO - ds_flow_evict : 0 2025-12-01 04:26:46,534 - INFO - ds_trap_original : 0 2025-12-01 04:26:46,534 - INFO - ds_no_memory : 0 2025-12-01 04:26:46,534 - INFO - ds_fragment_queue_fail : 0 2025-12-01 04:26:46,534 - INFO - ds_no_frag_entry : 0 2025-12-01 04:26:46,534 - INFO - ds_icmp_error : 0 2025-12-01 04:26:46,534 - INFO - ds_clone_fail : 0 2025-12-01 04:26:46,534 - INFO - ds_drop_pkts : 8 2025-12-01 04:26:46,534 - INFO - ---------------------------------------------------------------------------------------------------- 2025-12-01 04:26:46,534 - INFO - ds_discard : 6 2025-12-01 04:26:46,534 - INFO - ds_pull : 0 2025-12-01 04:26:46,534 - INFO - ds_invalid_if : 0 2025-12-01 04:26:46,534 - INFO - ds_invalid_arp : 0 2025-12-01 04:26:46,534 - INFO - ds_trap_no_if : 0 2025-12-01 04:26:46,534 - INFO - ds_nowhere_to_go : 0 2025-12-01 04:26:46,534 - INFO - ds_flow_queue_limit_exceeded : 0 2025-12-01 04:26:46,534 - INFO - ds_flow_no_memory : 0 2025-12-01 04:26:46,534 - INFO - ds_flow_invalid_protocol : 0 2025-12-01 04:26:46,534 - INFO - ds_flow_nat_no_rflow : 0 2025-12-01 04:26:46,534 - INFO - ds_flow_action_drop : 21 2025-12-01 04:26:46,534 - INFO - ds_flow_action_invalid : 0 2025-12-01 04:26:46,535 - INFO - ds_flow_unusable : 0 2025-12-01 04:26:46,535 - INFO - ds_flow_table_full : 0 2025-12-01 04:26:46,535 - INFO - ds_interface_tx_discard : 0 2025-12-01 04:26:46,535 - INFO - ds_interface_drop : 0 2025-12-01 04:26:46,535 - INFO - ds_duplicated : 0 2025-12-01 04:26:46,535 - INFO - ds_push : 0 2025-12-01 04:26:46,535 - INFO - ds_ttl_exceeded : 0 2025-12-01 04:26:46,535 - INFO - ds_invalid_nh : 5 2025-12-01 04:26:46,535 - INFO - ds_invalid_label : 0 2025-12-01 04:26:46,536 - INFO - ds_invalid_protocol : 0 2025-12-01 04:26:46,536 - INFO - ds_interface_rx_discard : 0 2025-12-01 04:26:46,536 - INFO - ds_invalid_mcast_source : 0 2025-12-01 04:26:46,536 - INFO - ds_head_alloc_fail : 0 2025-12-01 04:26:46,536 - INFO - ds_pcow_fail : 0 2025-12-01 04:26:46,536 - INFO - ds_mcast_clone_fail : 0 2025-12-01 04:26:46,536 - INFO - ds_rewrite_fail : 0 2025-12-01 04:26:46,536 - INFO - ds_misc : 0 2025-12-01 04:26:46,536 - INFO - ds_invalid_packet : 0 2025-12-01 04:26:46,536 - INFO - ds_cksum_err : 0 2025-12-01 04:26:46,536 - INFO - ds_no_fmd : 0 2025-12-01 04:26:46,536 - INFO - ds_invalid_vnid : 0 2025-12-01 04:26:46,537 - INFO - ds_frag_err : 0 2025-12-01 04:26:46,537 - INFO - ds_invalid_source : 0 2025-12-01 04:26:46,537 - INFO - ds_mcast_df_bit : 0 2025-12-01 04:26:46,537 - INFO - ds_l2_no_route : 2 2025-12-01 04:26:46,537 - INFO - ds_vlan_fwd_tx : 0 2025-12-01 04:26:46,537 - INFO - ds_vlan_fwd_enq : 0 2025-12-01 04:26:46,537 - INFO - ds_drop_new_flow : 0 2025-12-01 04:26:46,537 - INFO - ds_flow_evict : 0 2025-12-01 04:26:46,537 - INFO - ds_trap_original : 0 2025-12-01 04:26:46,537 - INFO - ds_no_memory : 0 2025-12-01 04:26:46,537 - INFO - ds_fragment_queue_fail : 0 2025-12-01 04:26:46,537 - INFO - ds_no_frag_entry : 0 2025-12-01 04:26:46,537 - INFO - ds_icmp_error : 0 2025-12-01 04:26:46,537 - INFO - ds_clone_fail : 0 2025-12-01 04:26:46,537 - INFO - ds_drop_pkts : 34 2025-12-01 04:26:46,537 - INFO - Deleting VM ctest-vm2-62768666 2025-12-01 04:26:46,631 - DEBUG - Verifying in api server 10.0.0.23 2025-12-01 04:26:46,631 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:46,643 - DEBUG - VM ID 83109cc2-bdc4-438e-9eba-07475b31157e of VM ctest-vm2-62768666 is still found in API Server 2025-12-01 04:26:48,644 - DEBUG - Verifying in api server 10.0.0.23 2025-12-01 04:26:48,644 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:48,651 - DEBUG - Response Code: 404 2025-12-01 04:26:48,651 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:48,659 - DEBUG - Response Code: 404 2025-12-01 04:26:48,659 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/83109cc2-bdc4-438e-9eba-07475b31157e 2025-12-01 04:26:48,666 - DEBUG - Response Code: 404 2025-12-01 04:26:48,667 - INFO - VM ctest-vm2-62768666 is fully removed in API-Server 2025-12-01 04:26:48,828 - DEBUG - Skipping VM-deletion-check in nova db since mysql_token is not found 2025-12-01 04:26:48,878 - DEBUG - Validated that vrouter 10.0.0.22 does not have VMs route for VN default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 2025-12-01 04:26:48,891 - DEBUG - Validated that vrouter 10.0.0.241 does not have VMs route for VN default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 2025-12-01 04:26:48,891 - INFO - Validated that all vrouters do not have VMs route for VN default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 2025-12-01 04:26:48,891 - INFO - VM ctest-vm2-62768666 is removed in Compute, and routes are removed in all compute nodes 2025-12-01 04:26:49,140 - INFO - Routes for VM ctest-vm2-62768666 is removed in all control-nodes 2025-12-01 04:26:49,141 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-2:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:26:49,195 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-2?flat 2025-12-01 04:26:49,228 - DEBUG - VM 83109cc2-bdc4-438e-9eba-07475b31157e is still present in vrouter an-jenkins-deploy-platform-ansible-os-4827-2 uve 2025-12-01 04:26:49,228 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:26:49,228 - DEBUG - interface for vm 83109cc2-bdc4-438e-9eba-07475b31157e deleted from vrouter uve ... 2025-12-01 04:26:49,228 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:26:52,229 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-2:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:26:52,281 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-2?flat 2025-12-01 04:26:52,317 - DEBUG - VM 83109cc2-bdc4-438e-9eba-07475b31157e is still present in vrouter an-jenkins-deploy-platform-ansible-os-4827-2 uve 2025-12-01 04:26:52,317 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:26:52,317 - DEBUG - interface for vm 83109cc2-bdc4-438e-9eba-07475b31157e deleted from vrouter uve ... 2025-12-01 04:26:52,317 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:26:55,318 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-2:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:26:55,381 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-2?flat 2025-12-01 04:26:55,417 - DEBUG - VM 83109cc2-bdc4-438e-9eba-07475b31157e is not present in vrouter an-jenkins-deploy-platform-ansible-os-4827-2 uve 2025-12-01 04:26:55,417 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:26:55,417 - DEBUG - interface for vm 83109cc2-bdc4-438e-9eba-07475b31157e deleted from vrouter uve ... 2025-12-01 04:26:55,418 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:26:55,418 - DEBUG - Validated that VM 83109cc2-bdc4-438e-9eba-07475b31157e is removed in Vrouter UVE 2025-12-01 04:26:55,418 - DEBUG - Verifying through opserver in 10.0.0.27 2025-12-01 04:26:55,418 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694?flat 2025-12-01 04:26:55,446 - DEBUG - 83109cc2-bdc4-438e-9eba-07475b31157e BM not in default-domain:ctest-TestDropStats-22981093:ctest-vn-18297694 uve 2025-12-01 04:26:55,446 - DEBUG - Validated that VM 83109cc2-bdc4-438e-9eba-07475b31157e is not present in the VN UVE 2025-12-01 04:26:55,446 - DEBUG - Verifying the 83109cc2-bdc4-438e-9eba-07475b31157e virtual network link through opserver 10.0.0.27 2025-12-01 04:26:55,446 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machines 2025-12-01 04:26:55,463 - INFO - 83109cc2-bdc4-438e-9eba-07475b31157e vm uve deleted from opserver 2025-12-01 04:26:55,464 - DEBUG - Verifying the 83109cc2-bdc4-438e-9eba-07475b31157e virtual network link through opserver 10.0.0.33 2025-12-01 04:26:55,464 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-12-01 04:26:55,476 - INFO - 83109cc2-bdc4-438e-9eba-07475b31157e vm uve deleted from opserver 2025-12-01 04:26:55,476 - DEBUG - Verifying the 83109cc2-bdc4-438e-9eba-07475b31157e virtual network link through opserver 10.0.0.23 2025-12-01 04:26:55,476 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2025-12-01 04:26:55,491 - INFO - 83109cc2-bdc4-438e-9eba-07475b31157e vm uve deleted from opserver 2025-12-01 04:26:55,491 - INFO - Validated that VM 83109cc2-bdc4-438e-9eba-07475b31157e is removed from Opserver 2025-12-01 04:26:55,491 - INFO - Deleting VM ctest-vm1-37313625 2025-12-01 04:26:55,577 - DEBUG - Verifying in api server 10.0.0.23 2025-12-01 04:26:55,577 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:55,587 - DEBUG - VM ID a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb of VM ctest-vm1-37313625 is still found in API Server 2025-12-01 04:26:57,588 - DEBUG - Verifying in api server 10.0.0.23 2025-12-01 04:26:57,588 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:57,597 - DEBUG - Response Code: 404 2025-12-01 04:26:57,597 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:57,605 - DEBUG - Response Code: 404 2025-12-01 04:26:57,605 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-machine/a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:26:57,613 - DEBUG - Response Code: 404 2025-12-01 04:26:57,613 - INFO - VM ctest-vm1-37313625 is fully removed in API-Server 2025-12-01 04:26:57,778 - DEBUG - Skipping VM-deletion-check in nova db since mysql_token is not found 2025-12-01 04:26:57,840 - DEBUG - Validated that vrouter 10.0.0.22 does not have VMs route for VN default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 2025-12-01 04:26:57,857 - DEBUG - Validated that vrouter 10.0.0.241 does not have VMs route for VN default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 2025-12-01 04:26:57,857 - INFO - Validated that all vrouters do not have VMs route for VN default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 2025-12-01 04:26:57,857 - INFO - VM ctest-vm1-37313625 is removed in Compute, and routes are removed in all compute nodes 2025-12-01 04:26:58,110 - INFO - Routes for VM ctest-vm1-37313625 is removed in all control-nodes 2025-12-01 04:26:58,110 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:26:58,164 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:26:58,198 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is still present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:26:58,198 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:26:58,199 - DEBUG - interface for vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb deleted from vrouter uve ... 2025-12-01 04:26:58,199 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:27:01,199 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:27:01,259 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:27:01,291 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is still present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:27:01,291 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:27:01,291 - DEBUG - interface for vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb deleted from vrouter uve ... 2025-12-01 04:27:01,291 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:27:04,292 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:27:04,350 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:27:04,391 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is still present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:27:04,391 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:27:04,391 - DEBUG - interface for vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb deleted from vrouter uve ... 2025-12-01 04:27:04,391 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:27:07,391 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:27:07,445 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:27:07,476 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is still present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:27:07,476 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:27:07,477 - DEBUG - interface for vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb deleted from vrouter uve ... 2025-12-01 04:27:07,477 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:27:10,477 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:27:10,530 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:27:10,566 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is still present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:27:10,566 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:27:10,566 - DEBUG - interface for vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb deleted from vrouter uve ... 2025-12-01 04:27:10,566 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:27:13,567 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:27:13,621 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:27:13,656 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is still present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:27:13,656 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:27:13,656 - DEBUG - interface for vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb deleted from vrouter uve ... 2025-12-01 04:27:13,656 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:27:16,657 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/generator/an-jenkins-deploy-platform-ansible-os-4827-1:Compute:contrail-vrouter-agent:0?flat 2025-12-01 04:27:16,711 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/vrouter/an-jenkins-deploy-platform-ansible-os-4827-1?flat 2025-12-01 04:27:16,742 - DEBUG - VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is not present in vrouter an-jenkins-deploy-platform-ansible-os-4827-1 uve 2025-12-01 04:27:16,742 - DEBUG - Verifying if the vm interface deleted from vroter uve... 2025-12-01 04:27:16,742 - DEBUG - interface for vm a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb deleted from vrouter uve ... 2025-12-01 04:27:16,742 - DEBUG - Deleted interface not in error interface list ... 2025-12-01 04:27:16,742 - DEBUG - Validated that VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is removed in Vrouter UVE 2025-12-01 04:27:16,743 - DEBUG - Verifying through opserver in 10.0.0.27 2025-12-01 04:27:16,743 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-network/default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141?flat 2025-12-01 04:27:16,764 - DEBUG - a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb BM not in default-domain:ctest-TestDropStats-22981093:ctest-vn-60352141 uve 2025-12-01 04:27:16,764 - DEBUG - Validated that VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is not present in the VN UVE 2025-12-01 04:27:16,764 - DEBUG - Verifying the a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb virtual network link through opserver 10.0.0.27 2025-12-01 04:27:16,764 - DEBUG - Requesting: http://10.0.0.27:8081/analytics/uves/virtual-machines 2025-12-01 04:27:16,778 - DEBUG - No links retuned for a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:27:16,778 - INFO - a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb vm uve deleted from opserver 2025-12-01 04:27:16,778 - DEBUG - Verifying the a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb virtual network link through opserver 10.0.0.33 2025-12-01 04:27:16,778 - DEBUG - Requesting: http://10.0.0.33:8081/analytics/uves/virtual-machines 2025-12-01 04:27:16,795 - DEBUG - No links retuned for a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:27:16,795 - INFO - a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb vm uve deleted from opserver 2025-12-01 04:27:16,795 - DEBUG - Verifying the a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb virtual network link through opserver 10.0.0.23 2025-12-01 04:27:16,795 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-machines 2025-12-01 04:27:16,809 - DEBUG - No links retuned for a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb 2025-12-01 04:27:16,809 - INFO - a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb vm uve deleted from opserver 2025-12-01 04:27:16,809 - INFO - Validated that VM a1e8db6a-cbd5-4c0b-91b3-d77c7e2888eb is removed from Opserver 2025-12-01 04:27:16,809 - INFO - Deleting VN ctest-vn-18297694 2025-12-01 04:27:17,028 - DEBUG - Response for deleting network () 2025-12-01 04:27:17,028 - INFO - Deleting VN ctest-vn-60352141 2025-12-01 04:27:17,277 - DEBUG - Response for deleting network () 2025-12-01 04:27:18,568 - DEBUG - Skipping xmpp flap check 2025-12-01 04:27:18,568 - INFO - END TEST : test_flow_action_drop_stats : PASSED[0:01:25] 2025-12-01 04:27:18,568 - INFO - -------------------------------------------------------------------------------- 2025-12-01 04:27:19,405 - INFO - Deleted project: ctest-TestDropStats-22981093, ID : eed38de3-3d15-4b55-9391-96e2d1d699ca