Can't override `onConnect` for subscriptions

[NickClark]

In order to properly do auth for subscriptions (the directive approach oddly doesn't work for subs), and configure the context in general, I need to be able to override it. This can be partially done by overriding it in the server settings. Unfortunately, when you add the need to reference this, it falls apart. Also, you need to replicate the existing method implementation, which is brittle and not likely to survive core library updates. Any advice on how to progress here? Seems like #73 might be related. Hopefully the fixes there will allow us to block the subscription at initialization since I've also noticed we can't seem to do much other than filter it lazily.

[icebob]

@Hugome could you help this issue?

[shawnmcknight]

I'm wondering if the changes I've recommended in #73 would alleviate the need to override the subscriptions property at all. Since the construction of the parameters would largely be falling to the ws action, could you solve your issue by using hooks or middlewares instead of overriding the onConnect handler?

[NickClark]

@shawnmcknight I believe so.

There are a few main stages I would like to consider in a normal subscription/WebSocket operation:

1. onConnect
   • This is where we Authenticate a user and would like to add the user to the Moleculer context initially
   • The same connection will then be used for multiple operations. For us, that includes Queries and Mutations.
2. Context
   • Apollo server creates a new context on every operation over the same connection. In our pre-Moleculer implementation, we use this opportunity to update the user permissions as extracted from the token. The GraphQL client includes the most recent token it has in the context of the operation call.
3. On subscription
   • When the initial subscription operation is made, it would be nice if we could pre-validate whether this is a valid subscription. Say a user tries to subscribe to a resource we know up-front it will never get events from due to the event filter, it would be nice to error out the operation. This isn't currently a deal-breaker for us but would be nice.
4. Event filter
   • Already handled in the current implementation

I'm currently working with a fork of moleculer-apollo-server. I should have more of a proposal today once I have something working there. If I understand correctly, you want to have an action, ws, that gets called to return, amongst other things, a context for onConnect? I'm a little unclear if we would just override it, or wrap it. But in general, I think that would help satisfy #1 above. I'm not sure exactly how #2 would be handled. #3 is still a question for me, since I think this may be an Apollo limitation. I have a custom @auth directive that works with other operations, just not subscriptions.

Thanks for everyone's attention to this. My efforts have been hampered by a lack of understanding in regard to how Moleculer ingresses operations and creates the initial context in general.

[NickClark]

Ok. After some more time looking into this, I've found that by simply adding a way to hook into both context and onConnect should do the trick.

this.apolloServer = new ApolloServer({
  schema,
  ..._.defaultsDeep({}, mixinOptions.serverOptions, {
    context: async (...args) => {
      const { req, connection } = args[0]
      let customContext;
      if (this.context) customContext = await this.context(...args);
      return req
        ? {
            ctx: req.$ctx,
            service: req.$service,
            params: req.$params,
            dataLoaders: new Map(), // create an empty map to load DataLoader instances into
          }
        : {
            service: connection.context.$service,
            ...customContext,
            // ctx: connection.context.$ctx
          };
    },
    subscriptions: {
      onConnect: async (connectionParams, socket) => {
        let customParams;
        if (this.onConnect) customParams = await this.onConnect(connectionParams, socket);
        return ({
          ...connectionParams,
          ...customParams,
          $service: this,
        });
      },
    },
  }),
});

Both are important. onConnect allows handing the connection when it is first instantiated, even before any operations are sent. context allows per-operation handling, like updating the token, on every operation. So they should not be bundled together. This simple modification helps if all I want to do is modify the eventual options being sent in the broker.call later. So I could just return {meta: {user: <myuser>}} and it should end up in the Moleculer context. The biggest improvement beyond this would be if we could get handed a Moleculer context right from the start, at onConnect. It sound like you've already suggested that, and I think would provide everything that we could need for now. As it stands, we don't have a context till the first action is called, which loses some of the context of where the action actually originiates.

[NickClark]

This seems to satisfy our needs:

createAsyncIteratorResolver(actionName, tags = [], filter) {
  return {
    subscribe: filter
      ? withFilter(
          () => this.pubsub.asyncIterator(tags),
          async (payload, params, graphqlCtx) => {
            // Use the context now provided in the graphql context. Make sure to not pass
            // Moleculer context to opts
            const { ctx, ...opts } = graphqlCtx;
            return payload !== undefined
              ? ctx.call(filter, { ...params, payload }, opts)
              : false;
          }
        )
      : () => this.pubsub.asyncIterator(tags),
    resolve: async (payload, params, graphqlCtx) => {
      // same as above.
      const { ctx, ...opts } = graphqlCtx;
      return ctx.call(actionName, { ...params, payload }, opts);
    },
  };
},

//......

this.apolloServer = new ApolloServer({
  schema,
  ..._.defaultsDeep({}, mixinOptions.serverOptions, {
    context: async (...args) => {
      const { req, connection } = args[0]
      // Allow custom contexts to be passed along
      let customContext;
      if (this.context) customContext = await this.context(...args);
      // Consitently pass context regardless of source
      return req
        ? {
            ctx: req.$ctx,
            service: req.$service,
            params: req.$params,
            dataLoaders: new Map(), // create an empty map to load DataLoader instances into
          }
        : {
            service: connection.context.$service,
            ...customContext,
            ctx: connection.context.$ctx,
          };
    },
    subscriptions: {
      onConnect: async (connectionParams, socket) => {
        // create a context right away.
        const $ctx = this.broker.ContextFactory.create(
          this.broker,
          null,
          {},
          {}
        );
        socket.$ctx = $ctx;
        // Allow custom contexts to be passed along
        let customParams;
        if (this.onConnect)
          customParams = await this.onConnect(
            connectionParams,
            socket
          );
        return {
          ...connectionParams,
          ...customParams,
          $service: this,
          $ctx,
        };
      },
    },
  }),
});

[NickClark]

In the above code, my biggest concern is how we are managing Moleculer context. Is it correct to create a context onConnect? Technically, every time context is called we are sourcing a new GraphQL operation. Should that be the source of a new Moleculer context? Should every context call also create a new Moleculer context for that operation, perhaps as a child of the Moleculer context created in onConnect?

[Hugome]

Joining @shawnmcknight, you will have a Moleculer context & more customization possibility with #73 and the new ws action.

[NickClark]

@shawnmcknight @Hugome I put this into Discord and then realized I should really add to the conversation here...

After taking a break to work on a different part of our app, I'm back on trying to get this working (which is now critical for us).

#73 was merged, but, unless I'm misunderstanding, it misses the per-operation case mentioned above.

I'm working on rewriting my example code to hook the new ws action to handle the onConnect case. Using authorize should be able to auth normal requests and upgraded ones.

Our code currently checks on every operation to see if the token we have is expired. We do this by hooking into Apollo's context thus allowing us to always check or update credentials as required. I think I may just be able to merge in my own implementation by copying molleculer-apollo's code and adding my own and overwriting the Apollo context function. But this is brittle. Is this the best way?

Using the new code, I'm seeing operations over that connection return a REQUEST_SKIPPED most of the time. Sometimes it works, but rarely. Does the context keep track of a time that is shared between operations?

[NickClark]

Ok... I'm pretty sure this isn't going to work. I think the context is maintaining an hrtime from when the ws action is first called at service startup... still testing some things. But I only seem to get operations to work over the same WebSocket for the first few seconds after startup. Goes back to my original thought... should we be creating new contexts per-operation?

[NickClark]

Ok.

Two issues remain to be fixed:

1. No hook for per-operation context manipulation or error handling.
2. Cannot handle using the same WebSocket past the configured requestTimeout period.
   I made a CodeSandbox example here

I'd like to work on a solution for this. But first I need to come up with an approach for handling the context. My thoughts are expressed above #75 (comment)

[NickClark]

@icebob Following on #75 (comment) and #75 (comment) I am trying to figure out what kind of context handing is preferable here.

It seems, for tracking purposes, we would want to make a context at the WebSocket connection time, and then a child context for each operation over that connection. But my main problem right now is that that will exceed the requestTimeout as tracked on the context.

What would be your desired approach from a design standpoint? (I'm still learning much about Moleculer's architecture and design choices, so I don't feel safe trying to implement anything yet)

[icebob]

Yeah, maybe creating a traced/tracked context is not a good option, if you never close it. Because you will lose the nested tracing for these calls (because the main parent span is never finished). Another problem is that if you use contextTracking (which collects the active contexts, and wait for them when you stop the broker), it also can cause a problem at stopping.

Sorry, but in the last months, I'm working totally different things, so I don't feel I understand the issue totally and I'm not using this module nowadays. Could you create a repo which contains only minimal things to demonstrate the problem and comments about what would be the desired "output"? Maybe it helps me to see the problem comprehensively.

[NickClark]

I already made a code sandbox showing the timeout issue above (Here's the link again).

But perhaps that shows an issue with the latest MR more than what I'd like to see happen... It does illustrate the desired operation. A single WebSocket connection, with support for multiple operations over that connection. Normally each operation/call/message would be equivalent to a single REST call. So it would make sense to track each one as such. But it would also be nice if we could somehow still map/track each call to it's parent WebSocket connection.

[NickClark]

@icebob @shawnmcknight @Hugome
Ok, take two... Here's another code sandbox. Both issues are resovled.

1. The context is now, like on WebSocket connect, handled via an action.
2. The startHrTime is now reset on every request.

Part two feels hacky. It still feels like we should be creating a new context for every operation.

Furthermore, it feels like we should have better traces. With the code above, a single WebSocket, then operation, will result in

┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ID: 84e879b6-da91-4689-93cb-421bd7c1957c                                       Depth: 1 Total: 1 │
├──────────────────────────────────────────────────────────────────────────────────────────────────┤
│ UPGRADE /graphql                                  2ms [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ID: af271eae-c3bc-4f0c-b4fc-209468c044b1                                       Depth: 1 Total: 1 │
├──────────────────────────────────────────────────────────────────────────────────────────────────┤
│ GQL context                                     158μs [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ID: 4708d7f6-226e-4335-83f2-65a192e3439e                                       Depth: 1 Total: 1 │
├──────────────────────────────────────────────────────────────────────────────────────────────────┤
│ action 'greeter.hello'                           84μs [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘

Where this would seem preferable:

┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ID: 84e879b6-da91-4689-93cb-421bd7c1957c                                       Depth: 1 Total: 1 │
├──────────────────────────────────────────────────────────────────────────────────────────────────┤
│ UPGRADE /graphql                                  2ms [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────────────────────────────────┐
│ ID: af271eae-c3bc-4f0c-b4fc-209468c044b1                                       Depth: 1 Total: 1 │
├──────────────────────────────────────────────────────────────────────────────────────────────────┤
│ GQL operation                                   158μs [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] │
│  └───  action 'greeter.hello'                    84μs [■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] │
└──────────────────────────────────────────────────────────────────────────────────────────────────┘

I plan to put what is in the code sandbox into an MR tomorrow morning... any feedback is appreciated. I'd really like to get the context and tracing stuff setup properly though.