Hammer commands throws error "undefined method `closed?' for nil:NilClass"
Environment
- Satellite 6.2.13
- Satellite 6.2.14
Issue
-
Hammer commands throws an error:
[root@satellite ~]# hammer host-collection list Error: undefined method `closed?' for nil:NilClass -
Trying hammer with debug logging on shows:
[root@satellite ~]# hammer -d host-collection list --organization=MyOrg [ INFO 2018-01-22 13:57:42 Init] Initialization of Hammer CLI (0.5.1.13) has started... [DEBUG 2018-01-22 13:57:42 Init] Running at ruby 2.2.2-p95 [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli_config.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/csv.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman_admin.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman_admin_logging_core.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman_admin_logging_katello.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman_bootdisk.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman_discovery.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman_docker.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman_remote_execution.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman_tasks.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/import.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /etc/hammer/cli.modules.d/katello.yml has been loaded [ INFO 2018-01-22 13:57:42 Init] Configuration from the file /root/.hammer/cli_config.yml has been loaded [DEBUG 2018-01-22 13:57:42 Connection] Registered: foreman [DEBUG 2018-01-22 13:57:42 API] Global headers: { :content_type => "application/json", :accept => "application/json;version=2", "Accept-Language" => "en" } [DEBUG 2018-01-22 13:57:42 API] Follow redirects: never [ INFO 2018-01-22 13:57:43 Modules] Extension module hammer_cli_foreman (0.5.1.10) loaded [ INFO 2018-01-22 13:57:43 Modules] Extension module hammer_cli_foreman_admin (0.0.5) loaded [ INFO 2018-01-22 13:57:43 Modules] Extension module hammer_cli_foreman_bootdisk (0.1.3.3) loaded [ INFO 2018-01-22 13:57:43 Modules] Extension module hammer_cli_foreman_discovery (0.0.2.3) loaded [ INFO 2018-01-22 13:57:43 Modules] Extension module hammer_cli_import (0.11.2) loaded [ INFO 2018-01-22 13:57:43 HammerCLI::MainCommand] subcommand organization (HammerCLIForeman::Organization) was removed. [ INFO 2018-01-22 13:57:43 HammerCLI::MainCommand] subcommand organization (HammerCLIKatello::Organization) was created. [ INFO 2018-01-22 13:57:44 Modules] Extension module hammer_cli_katello (0.0.22.29) loaded [DEBUG 2018-01-22 13:57:44 Init] Using locale 'en' [DEBUG 2018-01-22 13:57:44 Init] 'mo' files for locale domain 'hammer-cli' loaded from '/opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.13/locale' [DEBUG 2018-01-22 13:57:44 Init] 'mo' files for locale domain 'hammer-cli-foreman' loaded from '/opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/locale' [DEBUG 2018-01-22 13:57:44 Init] 'mo' files for locale domain 'hammer-cli-csv' loaded from '/opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_csv-2.2.1.1/locale' [DEBUG 2018-01-22 13:57:44 Init] 'mo' files for locale domain 'hammer_cli_foreman_docker' loaded from '/opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman_docker-0.0.6/locale' [DEBUG 2018-01-22 13:57:44 Init] 'mo' files for locale domain 'hammer-cli-katello' loaded from '/opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_katello-0.0.22.29/locale' [ INFO 2018-01-22 13:57:44 HammerCLI::MainCommand] Called with options: {"option_debug"=>true} [ INFO 2018-01-22 13:57:44 HammerCLIKatello::HostCollection] Called with options: {} [ INFO 2018-01-22 13:57:44 HammerCLIKatello::HostCollection::ListCommand] Called with options: {"option_organization_name"=>"MyOrg"} [ INFO 2018-01-22 13:57:44 API] Server: https://satellite.example.com/ [ INFO 2018-01-22 13:57:44 API] GET /katello/api/organizations [DEBUG 2018-01-22 13:57:44 API] Params: { :search => "name = \"MyOrg\"" } [DEBUG 2018-01-22 13:57:44 API] Headers: { :params => { :search => "name = \"MyOrg\"" } } [ERROR 2018-01-22 13:57:44 Exception] Error: undefined method `closed?' for nil:NilClass Error: undefined method `closed?' for nil:NilClass [ERROR 2018-01-22 13:57:44 Exception] NoMethodError (undefined method `closed?' for nil:NilClass): /opt/rh/rh-ruby22/root/usr/share/ruby/net/http/response.rb:329:in `stream_check' /opt/rh/rh-ruby22/root/usr/share/ruby/net/http/response.rb:199:in `read_body' /opt/rh/rh-ruby22/root/usr/share/ruby/net/http/response.rb:226:in `body' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.16/lib/apipie_bindings/api.rb:375:in `process_data' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.16/lib/apipie_bindings/api.rb:229:in `rescue in http_call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.16/lib/apipie_bindings/api.rb:220:in `http_call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.16/lib/apipie_bindings/api.rb:174:in `call_action' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.16/lib/apipie_bindings/api.rb:169:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.16/lib/apipie_bindings/resource.rb:14:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/id_resolver.rb:186:in `resolved_call' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/id_resolver.rb:172:in `find_resource_raw' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/id_resolver.rb:166:in `find_resource' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/id_resolver.rb:143:in `get_id' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/id_resolver.rb:133:in `block (2 levels) in define_id_finders' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:154:in `get_resource_id' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:223:in `block in customized_options' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:220:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:220:in `customized_options' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:244:in `request_params' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.13/lib/hammer_cli/apipie/command.rb:43:in `send_request' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:190:in `send_request' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:280:in `send_request' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:315:in `retrieve_all' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.10/lib/hammer_cli_foreman/commands.rb:299:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.13/lib/hammer_cli/abstract.rb:24:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/subcommand/execution.rb:11:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.13/lib/hammer_cli/abstract.rb:24:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/subcommand/execution.rb:11:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.13/lib/hammer_cli/abstract.rb:24:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:133:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.13/bin/hammer:125:in `' /bin/hammer:23:in `load'
Resolution
-
Rerun Satellite installer
# satellite-installer --scenario satellite --upgrade -
Restart Service
# katello-service restart -
Check network configuration to ensure hammer can connect to Satellite loopback interface on ports 80/443
-
If above steps did not help then check ENV variables for proxy as-
# env | grep -i proxy HTTPS_PROXY=https://<proxy-ip>:<proxy-port> HTTP_PROXY=http://<proxy-ip>:<proxy-port>
Note: un-setting above HTTPS_PROXY and HTTP_PROXY environment variables will allow hammer to communicate to the Satellite host.
For more KB articles/solutions related to Red Hat Satellite 6.x hammer Issues, please refer to the Consolidated Troubleshooting Article for Red Hat Satellite 6.x hammer-related Issues
Root Cause
-
Hammer is a front end for Satellite's REST API. It makes request to the REST API using the Satellite server's loopback interface. If for some reason, hammer cannot acquire the @socket, @socket will have nil type which throws the above exception.
-
There are multiple potential causes for this issue, such as:
- The satellite-installer failed
- Improper proxy configuration prevents hammer from acquiring socket on loopback interface
- Improper firewall configuration prevents hammer from acquiring socket on loopback interface
- Improper SSL configuration prevents hammer from acquiring socket on loopback interface
- HTTPS_PROXY/HTTP_PROXY environment variable is set due to which hammer is unable to communicate to Satellite
Diagnostic Steps
- Satellite installer log
/var/log/foreman-installer/satellite.logshows errors like:
[DEBUG 2018-01-16 11:07:36 main] Hook /usr/share/katello-installer-base/hooks/post/10-post_install.rb returned nil
[ INFO 2018-01-16 11:07:36 main] pulp.conf is already present, skipping
[DEBUG 2018-01-16 11:07:36 main] Hook /usr/share/katello-installer-base/hooks/post/29-create_package_httpd_conf.rb returned ["ssl.conf", "pulp.conf"]
[ERROR 2018-01-16 11:07:36 main] Upgrade failed during the installation phase. Fix the error and re-run the upgrade.
[DEBUG 2018-01-16 11:07:36 main] Hook /usr/share/katello-installer-base/hooks/post/30-upgrade.rb returned [<Logging::Logger:0x154fdfc name="main">, <Logging::Logger:0x1555220 name="fatal">]
[ INFO 2018-01-16 11:07:36 main] All hooks in group post finished
[DEBUG 2018-01-16 11:07:36 main] Exit with status code: 6 (signal was 6)
[ERROR 2018-01-16 11:07:36 main] Errors encountered during run:
[ERROR 2018-01-16 11:07:36 main] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[satellite.example.com]: Could not evaluate: Proxy satellite.example.com cannot be registered (undefined method `closed?' for nil:NilClass): N/A
[ERROR 2018-01-16 11:07:36 main] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[satellite.example.com]: Failed to call refresh: Proxy satellite.example.com cannot be registered (Proxy satellite.example.com cannot be registered (undefined method `closed?' for nil:NilClass): N/A): N/A
[ERROR 2018-01-16 11:07:36 main] /Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[satellite.example.com]: Proxy satellite.example.com cannot be registered (Proxy satellite.example.com cannot be registered (undefined method `closed?' for nil:NilClass): N/A): N/A
[ERROR 2018-01-16 11:07:36 main] Report processor failed: Could not send report to Foreman at https://satellite.example.com/api/reports: 502 "Bad Gateway"
This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.