Host Upgrade Install Fails When Attempting To Download hostagent.conf From Management Plane When Proxy Settings Are In Place.

Problem

Host upgrade install fails when attempting to download hostagent.conf from management plane. The proxy settings are in place. Post this failure, /var/opt/pf9/hostagent/XXXXX/desired_apps.json.part file and the /var/opt/pf9/hostagent/ directory itself is not seen. Restarting pf9-hostagent service recreates this directory back and then on a subsequent retry, host upgrade succeeds.

Hostagent logs
    
 
2023-05-05 18:26:36,376 - pf9_app_cache.py INFO - Using previously cached file /var/cache/pf9apps/pf9-hostagent/5.6.4-1078.9f795fe/pf9-hostagent-5.6.4-1078.9f795fe.x86_64.deb2023-05-05 18:26:36,377 - pf9_app.py INFO - Updating pf9-hostagent.5.6.4-1078.9f795fe2023-05-05 18:27:45,607 - pf9_app_db.py ERROR - Install command failed: sudo /opt/pf9/hostagent/bin/pf9-apt install /var/cache/pf9apps/pf9-hostagent/5.6.4-1078.9f795fe/pf9-hostagent-5.6.4-1078.9f795fe.x86_64.deb. Return code: 1, stdout: (Reading database ... 178992 files and directories currently installed.)Preparing to unpack .../pf9-hostagent-5.6.4-1078.9f795fe.x86_64.deb ...Downloading hostagent.conf from PF9Failed to download hostagent.conf from PF9, stderr: dpkg: error processing archive /var/cache/pf9apps/pf9-hostagent/5.6.4-1078.9f795fe/pf9-hostagent-5.6.4-1078.9f795fe.x86_64.deb (--install):new pf9-hostagent package pre-installation script subprocess returned error exit status 1Errors were encountered while processing:/var/cache/pf9apps/pf9-hostagent/5.6.4-1078.9f795fe/pf9-hostagent-5.6.4-1078.9f795fe.x86_64.debFailed to install package: /var/cache/pf9apps/pf9-hostagent/5.6.4-1078.9f795fe/pf9-hostagent-5.6.4-1078.9f795fe.x86_64.debTraceback (most recent call last):Traceback (most recent call last):File "/opt/pf9/hostagent/bin/pf9-apt", line 94, in <module>_install_package(package, cache)File "/opt/pf9/hostagent/bin/pf9-apt", line 72, in _install_package package_pathException: Errors encountered while installing : /var/cache/pf9apps/pf9-hostagent/5.6.4-1078.9f795fe/pf9-hostagent-5.6.4-1078.9f795fe.x86_64.deb2023-05-05 18:27:45,608 - session.py ERROR - Updating the pf9 host agent failedTraceback (most recent call last):File "/opt/pf9/hostagent/lib/python3.9/site-packages/bbslave/session.py", line 441, in update_agentprocess_agent_update(agent_info, agent_app_db, app_cache,File "/opt/pf9/hostagent/lib/python3.9/site-packages/pf9app/algorithms.py", line 216, in process_agent_update new_agent.update()File "/opt/pf9/hostagent/lib/python3.9/site-packages/pf9app/pf9_app.py", line 437, in updateself.app_db.update_package(local_path)File "/opt/pf9/hostagent/lib/python3.9/site-packages/pf9app/pf9_app_db.py",line 342, in update_packageself.pkgmgr.update_from_file(path)File "/opt/pf9/hostagent/lib/python3.9/site-packages/pf9app/pf9_app_db.py",line 116, in update_from_fileself.install_from_file(pkg_path)File "/opt/pf9/hostagent/lib/python3.9/site-packages/pf9app/pf9_app_db.py",line 106, in install_from_fileraise InstallOperationFailed()pf9app.exceptions.InstallOperationFailed2023-05-05 18:27:47,767 - session.py ERROR - Failed to save desired configuration: [Errno 2] No such file or directory: '/var/opt/pf9/hostagent/04755078-1580-4f3e-b886-d2e756d27ff0/desired_apps.json.part'2023-05-05 18:27:47,769 - session.py INFO - Already converged. Idling...2023-05-05 18:27:50,378 - session.py ERROR - timeout 120 /opt/pf9/hostagent/extensions/fetch_pod_info.py command failed: Command '['timeout', '120', '/opt/pf9/hostagent/extensions/fetch_pod_info.py']' returned non-zero exit status 1.
Copy

Environment

Platform9 Managed Kubernetes - v5.5

Answer

This is a known issue and Platform9 Engineering team is currently working on the CORE-1288.

Workaround

Try removing proxy environment variables from /etc/environment for worker nodes and added localhost to NO_PROXY variable on master nodes. Post doing this, host upgrades succeeds for most nodes.
Restart pf9-hostagent service.
Manually copy the .deb package from the active node and save it in the same location in the affected. Example: /var/cache/pf9apps/pf9-hostagent/5.6.4-1078.9f795fe/pf9-hostagent-5.6.4-1078.9f795fe.x86_64.deb
Restart the Platform9 services related to Hostagent, Comms and Sidekick.

Additional Information

To track the progress of the fix for this issue, open a support ticket mentioning the Jira CORE-1288.

Last updated on

Was this page helpful?