[Apps] Apps requiring access to podlogs, fail on bluefin (access denied)
Description
Problem/Justification
Impact
Activity

Automation for Jira February 9, 2023 at 3:27 PM
This issue has now been closed. Comments made after this point may not be viewed by the TrueNAS Teams. Please open a new issue if you have found a problem or need to re-engage with the TrueNAS Engineering Teams.

Waqar Ahmed February 9, 2023 at 3:27 PM
The symlink part has been fixed already, however after internal discussion we are not open to allowing apps to access /var/log as they could get sensitive information about the host which we definitely would not like. If the user is still interested in that, he/she can then in that case disable host path validation in apps settings.

Kjeld Schouten-lebbing January 23, 2023 at 9:05 PM
The issue has to do with complicated symlinks/cross-references and how hostPath validation validates those.
You don’t need to describe it, really. We know.
Even more so, because you do not really get what the underlaying issue is from a technical perspective. Which is fine, but it’s best to leave these issues to the people either developing the OS or the Apps in those cases
”in the case mentioned, this is read-only access.”
RO or RW is not of relevance here, hostPath validation cannot (and should not, for k8s security reasons, as it’s a HUGE vuln) differentiate between the two.
”I wouldn’t expect a read claim to be blocking….”
It’s not the claim that is blocking, it’s the symlinked path that is prohibited.
”would dataset migration fail?”
Zero relevance to the issue, like: It’s like you started talking about how green your grass is. ZERO relevance at all.
“I’d think that the WORST that’d happen (in this case) is promtail would end up watching a broken file descriptor/inode and the pod logs wouldn’t get pushed into prometheus….”
No of relevance, the actuall path is not allowed to be mounted for (very important) security reasons and is pseudo-randomised.
”I’m legitimately asking, as I dunno, but it doesn’t intuitively feel like it should cause a failure…”
I understand you’re asking, but this is a bugtracker for developers… not a Q&A or Educational program.
”I could absolutely see the concern if this was a write lock / claim… as I can see that preventing something from moving the dataset around…. but I didn’t think RO access would break anything.”
Moving the dataset around is not the issue and isn’t happening.
Write or Read is also completely not of any relevance to the issue.
So my two cents:
Please leave this issue for what it is, asking questions without thorough knowhow of the middleware and how kubernetes works, is basically going to waste a lot of developer time that is better put elsewhere. For example: I actually needed to postepone an important PR review to even write this.

Wolf Noble January 23, 2023 at 8:51 PM
- in the case mentioned, this is read-only access.
( specifically promtail is reading /var/log/pods/* and pushing bits of useful into prometheus )
I wouldn’t expect a read claim to be blocking….
would dataset migration fail?
I’d think that the WORST that’d happen (in this case) is promtail would end up watching a broken file descriptor/inode and the pod logs wouldn’t get pushed into prometheus….
I’m legitimately asking, as I dunno, but it doesn’t intuitively feel like it should cause a failure…
I could absolutely see the concern if this was a write lock / claim… as I can see that preventing something from moving the dataset around…. but I didn’t think RO access would break anything.

Kjeld Schouten-lebbing December 22, 2022 at 4:23 PM
It’s a pretty niche issue, but its awesome to see you picking it up so quickly
Lets look back into this after
Details
Details
Assignee

Reporter

Background
There are apps to parse pod/container logs for kubernetes, such as the populair Loki+Promtail combination.
These Apps are usually a tad more tricky to deploy, as they, obviously, require more access to information than the average App.
Normal Behavior
In this case the Apps try to read the App logs from
/var/log/pods
which is a carefully whitelisted path in the hostPath validation logic and should work fine.The bug
Sadly enough,
/var/log/pods
seems to be just a link to another path, as promtail/loki throw the following error:Error: Error response from daemon: invalid volume specification: '/var/log/pods:/var/log/pods:ro': /var/db/system/syslog-535a85bf35cc472d88ee664a1c80b367/log/pods path (real path of /var/log/pods) not allowed to be mounted
`The error speaks for itself:
/var/log/pods
links to another path that cannot be mounted as hostPathThe good news
It’s very good to see hostPath validation correctly prevent, what should be considered an escape out-of allowed hostPaths :)