FairCallQueue S3G impersonation on Ozone

Christos Bisias

Known issues

• When using Hadoop RPC with the FairCallQueue enabled
• All RPC accesses through S3G are recorded as S3G special-user(secure env)
• There is no S3G impersonation
• FairCallQueue isn’t making any distinction between S3G users
• It’s not working for S3G

IdentityProvider

• This is an interface used by the DecayRpcScheduler
• We can create our own implementation and define it in the config file
• UserIdentityProvider is the default implementation

public class UserIdentityProvider implements IdentityProvider {

public UserIdentityProvider() {

}

​

public String makeIdentity(Schedulable obj) {

UserGroupInformation ugi = obj.getUserGroupInformation();

return ugi == null ? null : ugi.getShortUserName();

}

}

​

Schedulable

• The Schedulable is an interface that defines what gets stored into the priority queues

public interface Schedulable {

UserGroupInformation getUserGroupInformation();

int getPriorityLevel();

}

​

OzoneIdentityProvider and new Schedulable

• Recap of the existing issue
• The Ozone S3G uses a permanent connection to the OM that masks the current user and all requests appear to be coming from the special S3G user
• The UGI stored in the Schedulable will always point to user “s3g”
• The Schedulable object needs to carry more information
• We can provide an Ozone side implementation of the IdentityProvider and define it on the config file
• OzoneIdentityProvider will access the new information on the Schedulable

CallerContext

• The CallerContext is a ThreadLocal variable that is passed from hadoop RPC client to server
• We can expose that field and use it to makeIdentity()
• It’s not used by Ozone
• It’s used by other hadoop applications for audit logging
• In Ozone, it could be repurposed to carry the identity info in a manner accessible to the IdentityProvider

Exposing the CallerContext

• We can expose the CallerContext in the Schedulable interface

​

public interface Schedulable {

UserGroupInformation getUserGroupInformation();

default public CallerContext getCallerContext() {

return null;

}

int getPriorityLevel();

}

Exposing the CallerContext

• On the ipc/Server class, we can modify the nested class Call that represents the RPC call we store on the FCQ
• Provide an accessor to the already available CallerContext

@Override

public CallerContext getCallerContext() {

return this.callerContext;

}

• This isn’t making any functional changes to Hadoop but only exposing existing data

Ozone side changes

• Implement OzoneIdentityProvider
• Check CallerContext and if it’s not null, return it

public String makeIdentity(Schedulable schedulable) {

UserGroupInformation ugi = schedulable.getUserGroupInformation();

CallerContext callerContext = schedulable.getCallerContext();

if (callerContext != null) {

if (!StringUtil.isNullOrEmpty(callerContext.getContext())) {

return callerContext.getContext();

}

}

return ugi.getShortUserName() == null ? null : ugi.getShortUserName();

}

Ozone side changes

• We need to set the CallerContext on the client side before the ipc/Server Listener picks up the request
• On OzoneManagerProtocolClientSideTranslatorPB.submitRequest()
• Set the CallerContext

CallerContext callerContext =

new CallerContext

.Builder(threadLocalS3Auth.get().getAccessID())

.build();

CallerContext.setCurrent(callerContext);

Would the hadoop team consider accepting such a change?

If so, what are the next steps?