object/GET: allow to fetch exact EC part in GET request#393
Conversation
Closes #348. Signed-off-by: Pavel Karpy <carpawell@nspcc.io>
carpawell
force-pushed
the
feat/EC-GET-with-no-part-index
branch
from
9c0e665 to
8fe6ee4
Compare
| // `PlacementPolicy.ec_rules` list. | ||
| // If `__NEOFS__EC_PART_IDX` is also included in X-headers, node returns | ||
| // corresponding part, otherwise nodes returns any part for the rule | ||
| // described above it has (if more that one part were found, the order |
There was a problem hiding this comment.
i find it dangerous when netmap is changing, and there could be policer's object changes. the same is about any form of GETRANGE
There was a problem hiding this comment.
Multipart can be solved by returning "appropriate" (as per current placement) or lower index (if there is no appropriate) part.
There was a problem hiding this comment.
Multipart can be solved by returning "appropriate"
from which side? client or server? what if they are different? if we believe that netmap is ok (not changing right now) why is this optimisation needed at all?
There was a problem hiding this comment.
Only server (the one processing the request) can decide here.
| // `__NEOFS__EC_PART_IDX` by object attribute format. Rule index MUST NOT | ||
| // exceed container's `PlacementPolicy.ec_rules` list. Part index MUST NOT | ||
| // exceed total part number in the indexed rule. | ||
| // - `meta_header.x_headers` includes `__NEOFS__EC_RULE_IDX` by object |
There was a problem hiding this comment.
why not do the same for GetRange?
There was a problem hiding this comment.
as i stated in the thread above: i am afraid that we will have many problems with undefined requests (no part index x-headers) returning objects if no header is received in the same request (any GETRANGE/GET with range and payload_only) is not safe cause it is impossible to know what exact object was handled
| // `PlacementPolicy.ec_rules` list. | ||
| // If `__NEOFS__EC_PART_IDX` is also included in X-headers, node returns | ||
| // corresponding part, otherwise nodes returns any part for the rule | ||
| // described above it has (if more that one part were found, the order |
There was a problem hiding this comment.
order is undefined
since parts are streamed to the client one-by-one, i think it is worth guaranteeing direct order. Is there any reason not to do so?
There was a problem hiding this comment.
do you suggest returning every part in the same GET request in case of any conflicts?
roman-khimov
left a comment
roman-khimov
left a comment
There was a problem hiding this comment.
The spec is rather simple, however this raises questions over forwarding optimizations, these require strict ordering and we can end up with more simultaneous connections.
| // `PlacementPolicy.ec_rules` list. | ||
| // If `__NEOFS__EC_PART_IDX` is also included in X-headers, node returns | ||
| // corresponding part, otherwise nodes returns any part for the rule | ||
| // described above it has (if more that one part were found, the order |
There was a problem hiding this comment.
Multipart can be solved by returning "appropriate" (as per current placement) or lower index (if there is no appropriate) part.
| // described above it has (if more that one part were found, the order | ||
| // is undefined). Part index MUST NOT exceed total part number in the | ||
| // indexed rule. `__NEOFS__EC_PART_IDX` MUST NOT be omitted with | ||
| // non-nil `body.range` or `body.payload_only` field set to `true`. |
There was a problem hiding this comment.
Range does require idx for sure, but payload only can be solved with x header returned from node.
There was a problem hiding this comment.
i vote for returning this header always then
3 participants
Closes #348.