Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retina agent is failing to initialize with "failed to reconcile plugin dropreason" #1069

Open
devkoriel opened this issue Nov 26, 2024 · 0 comments
Assignees

Comments

@devkoriel
Copy link

devkoriel commented Nov 26, 2024

Describe the bug
Retina agent is failing to initialize with "failed to reconcile plugin dropreason"

To Reproduce
Steps to reproduce the behavior:

  1. Install retina with helm upgrade with operator.
  2. enabledPlugin_linux: '["dropreason","packetforward","linuxutil","dns","tcpretrans"]'

Expected behavior
Retina agent starts successfully.

Screenshots

ts=2024-11-26T03:26:06.292Z level=debug caller=loader/compile.go:26 msg="Error running command" command="/usr/bin/clang -target bpf -Wall -D__TARGET_ARCH_x86 -g -O2 -c /go/src/github.com/microsoft/retina/pkg/plugin/dropreason/_cprog/drop_reason.c -o /go/src/github.com/microsoft/retina/pkg/plugin/dropreason/kprobe_bpf.o -I/go/src/github.com/microsoft/retina/pkg/plugin/dropreason/../lib/_amd64 -I/go/src/github.com/microsoft/retina/pkg/plugin/dropreason/../lib/common/libbpf/_src -I/go/src/github.com/microsoft/retina/pkg/plugin/dropreason/../filter/_cprog/" stderr= error="signal: killed"
ts=2024-11-26T03:26:06.292Z level=info caller=conntrack/conntrack_linux.go:79 msg="Stopping conntrack GC loop"
ts=2024-11-26T03:26:06.292Z level=info caller=server/server.go:79 msg="gracefully shutting down HTTP server..."
ts=2024-11-26T03:26:06.292Z level=info caller=watchermanager/watchermanager.go:71 msg="watcher stopping..." watcher_type=*apiserver.ApiServerWatcher
ts=2024-11-26T03:26:06.292Z level=info caller=watchermanager/watchermanager.go:71 msg="watcher stopping..." watcher_type=*endpoint.EndpointWatcher
ts=2024-11-26T03:26:06.293Z level=info caller=server/server.go:71 msg="HTTP server stopped with err: http: Server closed"
ts=2024-11-26T03:26:06.293Z level=panic caller=controllermanager/controllermanager.go:119 msg="Error running controller manager" error="failed to reconcile plugin dropreason: failed to compile plugin: unable to compile eBPF code: signal: killed" errorVerbose="signal: killed\nunable to compile eBPF code\ngithub.com/microsoft/retina/pkg/plugin/dropreason.(*dropReason).Compile\n\t/go/src/github.com/microsoft/retina/pkg/plugin/dropreason/dropreason_linux.go:112\ngithub.com/microsoft/retina/pkg/managers/pluginmanager.(*PluginManager).Reconcile\n\t/go/src/github.com/microsoft/retina/pkg/managers/pluginmanager/pluginmanager.go:107\ngithub.com/microsoft/retina/pkg/managers/pluginmanager.(*PluginManager).Start\n\t/go/src/github.com/microsoft/retina/pkg/managers/pluginmanager/pluginmanager.go:164\ngithub.com/microsoft/retina/pkg/managers/controllermanager.(*Controller).Start.func1\n\t/go/src/github.com/microsoft/retina/pkg/managers/controllermanager/controllermanager.go:109\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1700\nfailed to compile plugin\ngithub.com/microsoft/retina/pkg/managers/pluginmanager.(*PluginManager).Reconcile\n\t/go/src/github.com/microsoft/retina/pkg/managers/pluginmanager/pluginmanager.go:108\ngithub.com/microsoft/retina/pkg/managers/pluginmanager.(*PluginManager).Start\n\t/go/src/github.com/microsoft/retina/pkg/managers/pluginmanager/pluginmanager.go:164\ngithub.com/microsoft/retina/pkg/managers/controllermanager.(*Controller).Start.func1\n\t/go/src/github.com/microsoft/retina/pkg/managers/controllermanager/controllermanager.go:109\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1700\nfailed to reconcile plugin dropreason\ngithub.com/microsoft/retina/pkg/managers/pluginmanager.(*PluginManager).Start\n\t/go/src/github.com/microsoft/retina/pkg/managers/pluginmanager/pluginmanager.go:168\ngithub.com/microsoft/retina/pkg/managers/controllermanager.(*Controller).Start.func1\n\t/go/src/github.com/microsoft/retina/pkg/managers/controllermanager/controllermanager.go:109\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:78\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1700"
panic: Error running controller manager

goroutine 61 [running]:
go.uber.org/zap/zapcore.CheckWriteAction.OnWrite(0x1?, 0x10?, {0x0?, 0x0?, 0xc001296380?})
	/go/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:196 +0x54
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000775c70, {0xc000e9a100, 0x1, 0x1})
	/go/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:262 +0x24e
go.uber.org/zap.(*Logger).Panic(0xc0005126c0?, {0x328c7d8?, 0x0?}, {0xc000e9a100, 0x1, 0x1})
	/go/pkg/mod/go.uber.org/[email protected]/logger.go:285 +0x51
github.com/microsoft/retina/pkg/managers/controllermanager.(*Controller).Start(0xc0005389b0, {0x371acb8?, 0xc000538690?})
	/go/src/github.com/microsoft/retina/pkg/managers/controllermanager/controllermanager.go:119 +0x27c
created by github.com/microsoft/retina/cmd/legacy.(*Daemon).Start in goroutine 1
	/go/src/github.com/microsoft/retina/cmd/legacy/daemon.go:315 +0x2c55

Platform (please complete the following information):

  • OS: Amazon Linux 2023.6.20241010
  • Kubernetes Version: v1.30.4-eks-a737599
  • Host: EKS
  • Retina Version: v0.0.18
  • Kernel Version: 6.1.112-122.189.amzn2023.x86_64
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

2 participants