Repository navigation
golink: escape the path remainder appended to non-template links - #269
abhiramshaji-cyber wants to merge 4 commits into
Conversation
serveGo passes the decoded request path remainder to expandLink, which appended it to the long URL and parsed the result. A remainder such as "100%" (from /who/100%25) made url.Parse fail and the request returned a 500, and a remainder containing "?" or "#" moved text out of the path and into the query or fragment of the redirect target. For links without a template, escape the remainder as a URL path before appending it, so characters such as "%", "?" and "#" stay part of the path. Templates still receive the decoded value in .Path. Fixes tailscale#255 Signed-off-by: abhiramshaji-cyber <write4abhiram@gmail.com>
|
Could a maintainer approve the workflow runs? The tests, Nix build, Docker and govulncheck jobs are waiting on approval before they start; this fixes #255. |
|
Hi @mikeodr, hope you're doing well! Hey, it's me again with this one; I can see you've been busy keeping the deps and govulncheck in good shape, so no rush at all. The workflow runs here are still waiting on approval, and I'm happy to rebase onto main so it picks up today's govulncheck fix, or reshape the change if you'd like it done differently. Out of curiosity, now that golink can be imported as a library (#252), are you seeing people embed it in their own tools, or is the standalone server still how most folks run it? Any tips for contributing here would be really appreciated too. |
|
If you could rebase that would be swell I can give this a review. I don't have any data on your embed vs not unfortunately. |
…remainder Signed-off-by: abhiramshaji-cyber <write4abhiram@gmail.com>
|
Hey @mikeodr, thank you so much, that's really kind! I merged |
mikeodr
left a comment
There was a problem hiding this comment.
Thanks for the PR, it's nearly there but I think a few small issues remain.
There was a problem hiding this comment.
concern(blocking): This is now out of sync with the new change. Can you unifi it with the new logic?
I believe also L1457 has the same concern.
There was a problem hiding this comment.
Hope you're doing well, and thanks for catching this, you're right that both spots were still cutting the decoded path. In 4155412 serveGo and resolveLink now both go through one small splitLinkPath, which cuts the escaped path at the first / and then decodes the name and the rest.
One side effect I wanted to check with you: since the split happens on the escaped path, /who%2Fextra is now a 404 where main redirected it to who with extra. I added a test for it in TestServeGo. Does that seem right to you, or would you rather keep the old behaviour there? Happy to change it either way.
| return buf.String(), nil | ||
| } | ||
|
|
||
| func appendPath(long, path string) string { |
There was a problem hiding this comment.
question: I believe appendPath ignores query and fragment in the long URL. With link g pointing to https://www.google.com/search?q=golink, request /g/foo%25 yields ...search?q=golink/foo%25, placing the remainder in the query value. Appending to the parsed URL's Path should fix this, and expandLink parses the result on the very next line anyway.
Can you confirm and add appropriate tests?
There was a problem hiding this comment.
Thanks a lot, confirmed, you were right. With https://www.google.com/search?q=golink the rest was landing inside the q value. In 4155412 expandLink parses the long URL first and appendPath adds the rest to u.Path (and u.RawPath when it's set), so /g/foo%25 now goes to https://www.google.com/search/foo%25?q=golink, and a fragment on the long URL stays at the end too.
Tests are in TestExpandLink (query, query plus fragment plus request query, a long URL with %2f) and in TestServeGo for the google style link. 6857f84 also fixes two cases I found while testing this: a long URL ending in %2F, and opaque ones like mailto:. Let me know if I missed a case you had in mind.
| if !strings.Contains(long, "{{") { | ||
| return appendPath(long, env.Path), nil | ||
| } | ||
| tmpl, err := texttemplate.New("").Funcs(expandFuncMap).Parse(long) |
There was a problem hiding this comment.
concern: Templated links still 500 on the issue #255 input. A link using {{.Path}} explicitly writes http://who/100% unescaped, url.Parse fails, and serveGo returns 500. The implicit default and the explicit template used to behave identically and now diverge. I think this should at minimum be a 4xx.
Please confirm and add tests as appropriate.
There was a problem hiding this comment.
Good catch, thank you, confirmed. A link like http://who/{{.Path}} with /tmpl/100%25 still fails url.Parse, since the template writes the decoded 100%. In 4155412 serveGo now returns 400 when the error from expandLink is a *url.Error, and keeps 401 for errNoUser.
I added a TestServeGo case for a templated link (302 for a plain path, 400 for %25), plus TestExpandLink cases showing {{PathEscape .Path}} handles 100% fine. One thing I'm not sure about: a saved link without a template whose long URL itself won't parse would also get 400 now instead of 500. Would you prefer I keep that one as 500? Happy to adjust.
| if !strings.HasSuffix(long, "/") { | ||
| long += "/" | ||
| } | ||
| return long + (&url.URL{Path: path}).EscapedPath() |
There was a problem hiding this comment.
concern(blocking): I think after running some tests there's a behaviour change.
# now encodes as %23, which changes go/doc-with-anchor used to be http://docs-server/something#anchor and is now http://docs-server/something%23anchor, which may or may not breaking jump to header anchor points on a page.
Please test and confirm.
There was a problem hiding this comment.
Thank you for testing this, you're totally right, that was a real behaviour change and I should have caught it. In 4155412 a # in the rest starts the fragment again: appendPath cuts the rest at the first # and sets u.Fragment from the part after it, so go/doc/something%23anchor goes to http://docs-server/something#anchor like before. A fragment already on the long URL is kept unless the rest brings its own.
Tests cover a#b, a lone #b, a trailing #, and a long URL with its own fragment in TestExpandLink, plus %23 in TestServeGo and TestResolveLink. Does that match what you saw?
| @@ -998,24 +998,12 @@ func regexMatch(pattern string, s string) bool { | |||
| // If long does not include templates, the default behavior is to append | |||
| // env.Path to long. | |||
There was a problem hiding this comment.
chore: I don't think this matches the new behaviour, can you ensure it is up to date?
There was a problem hiding this comment.
Thanks, good point, it was out of date. In 4155412 the expandLink comment now says templates run with env, and otherwise the path is escaped onto the long URL's path with a # starting the fragment. Happy to reword it if you'd like it shorter or closer to the old style.
serveGo and resolveLink now split the escaped request path with one shared helper, so the short name and remainder come from the same escaping rules. For links without a template, the remainder is appended to the parsed long URL's Path (and RawPath when set), so a long URL with a query or fragment keeps them after the new path. A "#" in the remainder starts the fragment of the redirect target, as it did before. A templated link whose output is not a valid URL now returns 400 instead of 500. Updates tailscale#255 Signed-off-by: Abhiram Shaji <write4abhiram@gmail.com>
Check for a trailing slash on the escaped path, so a long URL ending in %2F still gets a "/" before the remainder, and append the remainder to Opaque for long URLs such as mailto:, where String ignores Path. Signed-off-by: Abhiram Shaji <write4abhiram@gmail.com>
|
Hi @mikeodr, hope you're doing well! Thank you so much for the careful review and for running it locally, the |
Fixes #255
Opening a go link with an encoded
%or?after the link name returned a 500 or redirected to the wrong address; now they stay in the redirect path, and#still starts the fragment.What this code is for
serveGohandles a request such ashttp://go/who/100%25. It splits the path into the link name (who) and the rest (100%), loads the link, and callsexpandLinkto build the redirect target.resolveLinkdoes the same split for a link given as text, such asgo/who/foo. For a link without a template,expandLinkadds the rest of the path to the long URL.Before this PR
A person has a link
searchthat points tohttps://www.google.com/search?q=golinkand openshttp://go/search/100%25.serveGocutsr.URL.Path, which Go has already decoded, so the rest is100%.expandLinksees no{{in the long URL and adds{{with .Path}}/{{.}}{{end}}to the end of the string.100%as is, givinghttps://www.google.com/search?q=golink/100%.url.Parserejects it withinvalid URL escape "%".%, the rest lands inside theqvalue, not the path.After this PR
Same person, same request.
serveGocallssplitLinkPath(r.URL.EscapedPath()), which cuts the escaped path at the first/and then decodes each part, so the rest is100%.resolveLinkcalls the same function.expandLinksees no{{and parses the long URL first.appendPathadds/100%tou.Path(and tou.RawPathwhen the long URL has one), so the queryq=golinkstays where it was.https://www.google.com/search/100%25?q=golink.#in the rest (from%23) is cut off and set asu.Fragment, sogo/doc/page%23anchorstill opens.../page#anchor.http://who/{{.Path}}, that writes100%still failsurl.Parse, and now gets a 400 instead of a 500.The diagram
flowchart LR classDef box fill:#eef1ff,stroke:#4c5bd4,color:#111827 classDef bad fill:#ffd6d6,stroke:#cc0000,color:#111827,stroke-width:3px classDef good fill:#c8f0c8,stroke:#008800,color:#111827,stroke-width:3px subgraph BEFORE["BEFORE: /search/100%25 returns 500"] direction TB B1["open go/search/100%25"] --> B2["serveGo: rest is 100%"] B2 --> B3["expandLink"] B3 --> B4{"add rest to end of long string"} B4 -->|"search?q=golink/100%"| B5["url.Parse error, 500"] B4 -->|"rest is extra"| B6["rest still lands in q value"] end subgraph AFTER["AFTER: redirect to /search/100%25?q=golink"] direction TB A1["open go/search/100%25"] --> A2["serveGo: splitLinkPath, rest is 100%"] A2 --> A3["expandLink parses long URL"] A3 --> A4{"THE CHANGE: appendPath adds rest to u.Path"} A4 -->|"rest is 100%"| A5["search/100%25?q=golink"] A4 -->|"rest is page#anchor"| A6["page#anchor, same as before"] end B1 ~~~ A1 class B1,B2,B3,B6 box class A1,A2,A3,A6 box class B4,B5 bad class A4,A5 good linkStyle 3 stroke:#cc0000,stroke-width:5px linkStyle 8 stroke:#008800,stroke-width:5px style BEFORE fill:#fff5f5,stroke:#cc0000,color:#111827 style AFTER fill:#f3fff3,stroke:#008800,color:#111827The change
In
golink.go:expandLinkruns the template only when the long URL has{{, then always callsurl.Parse. For a link without a template it then callsappendPath. This is the fix.appendPathcuts the rest at the first#. The part after it, when not empty, replacesu.Fragment. The part before it is added tou.Path, with a/in between unless the escaped path already ends in one, and the escaped form is added tou.RawPathwhen that is set, so a long URL such ashttp://host.com/foo%2f/barkeeps its%2f. For a long URL such asmailto:a@b.com, the rest goes ontou.Opaqueinstead.splitLinkPathis the one place that cuts a path into link name and rest, used by bothserveGoandresolveLink.serveGoreturns 400 when the error fromexpandLinkis a*url.Error, and still returns 401 forerrNoUser.golink_test.goadds rows toTestExpandLink(%,?,#, a lone#, a trailing#, spaces, unicode, a trailing slash, long URLs with a query and a fragment, a long URL withRawPath, and templates with100%), toTestServeGo(%25,%3F,%23, unicode, a long URL with a query, and a templated link that now gets 400), and toTestResolveLink(%25and%23).What changes for a user
go/who/100%25redirects tohttp://who/100%25instead of returning a 500.go/who/a%3Fbkeeps?in the path instead of starting a query.go/who/a%23bstill redirects tohttp://who/a#b, so links to a heading on a page keep working.go/who/a/b, request query strings, and template functions such as{{PathEscape .Path}}behave as before. A%2Fin the rest is still decoded to/, as before. A%2Fin the link name is no longer treated as a/, sogo/who%2Fextrais a 404 wheremainredirected it.