3.2.2 to 3.3.0

Summary

Optimization Server 3.3.0 introduces break changes in:

It also deprecates application properties used in the workers (both Java and Python)

You are concerned by this guide if :

Development of applications

Library Backward compatibility Deprecations
Master API client Break changes: adjust code
Worker (Java) Break changes: adjust code Deprecations
Worker (Python) Backward compatible Deprecations

Deployment of Optimization Server

Chart Backward compatibility Deprecations
dbos-volume Backward compatible
dbos-secrets Backward compatible
dbos-infra Backward compatible
dbos Backward compatible
cplex Backward compatible Deprecations
wod Backward compatible Deprecations

Master API client

For the synchronous client, please read this section

For the asynchronous clients, read:

The synchronous API client

The former Java clients:

  • jersey
  • rest template
  • feign

have been merged into one single optimserver-api-client Java library.

Optimization Server 3.3.0 also provides a Spring Boot starter module if you want to integrate the library in a Spring Boot application.

Read this section for mode details about the new Java library.

Read this one for mode details about the Spring integration.

Break changes: adjust code

Even though most of the code from the former clients is still functional, some of it must be adjusted.

The import statements

The sub-packages jersey2, feign and resttemplate have disappeared.

3.2.2:

import com.decisionbrain.optimserver.client.java.jersey2.ApiClient;
import com.decisionbrain.optimserver.client.java.feign.ApiClient;
import com.decisionbrain.optimserver.client.java.resttemplate.ApiClient;

import com.decisionbrain.optimserver.client.java.jersey2.model.JobStatusEvent;
import com.decisionbrain.optimserver.client.java.feign.model.JobStatusEvent;
import com.decisionbrain.optimserver.client.java.resttemplate.model.JobStatusEvent;

3.3.0:

import com.decisionbrain.optimserver.client.java.ApiClient;
import com.decisionbrain.optimserver.master.model.JobStatusEvent;

The ‘File’ type

The APIs that used File as the return type now return InputStream

3.2.2:

BucketApi api;
String bucketId = "id";
...
File bucketFile = api.getBucketContent(bucketId);
try(FileInputStream fileInputStream = new FileInputStream(bucketFile)){
    byte[] bucketData = fileInputStream.readAllBytes();
}

3.3.0:

BucketApi api;
String bucketId = "id";
....
try(InputStream bucketStream = api.getBucketContent(bucketId)){
    byte[] bucketData = bucketStream.readAllBytes();
}

The Spring AMQP Asynchronous API client

The Asynchronous API has been fully rewritten in Optimization Server 3.3.0.

3.2.2:

implementation "com.decisionbrain:optimserver-client-amqp-spring:3.2.2"

@SpringBootApplication
@EnableOptimServerAmqpClient
public class ApiAmqpClientApplication {
    public static void main(String[] args) {
        SpringApplication.run(ApiAmqpClientApplication.class, args);
    }
}

import com.decisionbrain.optimserver.client.amqp.service.JobsCompletionWaitingService;
import com.decisionbrain.optimserver.client.amqp.service.JobEventsStandaloneSubscriptionService;
import com.decisionbrain.optimserver.client.java.jersey2.api.JobExecutionApi;

@Service
public class MyService
{
    private final JobsCompletionWaitingService jobsCompletionWaitingService;
    private final JobEventsStandaloneSubscriptionService jobEventsListenerRegisteringService;
    private final JobExecutionApi jobExecutionClient;

    public MyService(JobsCompletionWaitingService jobsCompletionWaitingService,
                     JobEventsStandaloneSubscriptionService jobEventsListenerRegisteringService,
                     JobExecutionApi jobExecutionClient) {
        this.jobsCompletionWaitingService = jobsCompletionWaitingService;
        this.jobEventsListenerRegisteringService = jobEventsListenerRegisteringService;
        this.jobExecutionClient = jobExecutionClient;
    }
    public void listenToOptimServerJobSolution() {
        String jobId = "created job Id";
        long timeout = 5000L;
        jobsCompletionWaitingService.waitForJobCompletion(jobId, timeout * 2, () -> {
            return jobExecutionClient.startAsyncJobExecution(jobId, timeout) != null;
        }).ifPresent((SolutionDTO solution) -> {
            LOGGER.info("Solution retrieved before timeout");
        });
    }

    void listenToOptimServerJobEvents() {
        String jobId = "created job Id";
        jobEventsListenerRegisteringService.registerJobEventsListener(jobId, new JobEventListener() {
            @Override
            public void onStatus(JobStatusDTO jobStatus) {
                LOGGER.info(String.format("[STATUS]: %s", jobStatus.getStatus().name()));
            }
        });
    }    
}

3.3.0:

implementation "com.decisionbrain:spring-boot-starter-optimserver-amqp-client:3.3.0"

@SpringBootApplication
@EnableOptimServerClient
public class ApiAmqpClientApplication {
    public static void main(String[] args) {
        SpringApplication.run(ApiAmqpClientApplication.class, args);
    }
}

import com.decisionbrain.optimserver.client.java.async.api.JobExecutionAsyncApi;
import com.decisionbrain.optimserver.client.java.async.api.JobEventSource;
import com.decisionbrain.optimserver.client.java.api.JobExecutionApi;

import java.time.Duration;

@Service
public class MyService {
    private final JobExecutionAsyncApi jobExecutionAsyncApi;
    private final JobExecutionApi jobExecutionClient;

    public MyService(JobExecutionAsyncApi jobExecutionAsyncApi, JobExecutionApi jobExecutionClient) {
        this.jobExecutionAsyncApi = jobExecutionAsyncApi;
        this.jobExecutionClient = jobExecutionClient;
    }
    
    public void listenToOptimServerJobSolution() {
        String jobId = "created job Id";
        long timeout = 5000L;
        jobExecutionAsyncApi.getJobSolution(jobId, Duration.ofMillis(timeout * 2))
                .whenComplete((jobSolution, throwable) -> {
            if (jobSolution != null) {
                LOGGER.info("Solution retrieved before timeout");
            }
        });
        jobExecutionApi.startAsyncJobExecution(jobDefinition.getId(), timeout);
    }

    void listenToOptimServerJobEvents() {
        String jobId = "created job Id";
        JobSubscriptionFilter filter = new JobSubscriptionFilter(jobId);
        JobEventSource eventSource = jobExecutionAsyncApi.getJobEventSource(filter);

        eventSource.statusEvents()
                .subscribe(new ConsumerSubscriber<>(
                jobStatusEvent -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()))
                ));
        eventSource.connect();
    }
}

The Spring SSE Asynchronous API client

The Asynchronous API has been fully rewritten in Optimization Server 3.3.0.

3.2.2:

implementation "com.decisionbrain:optimserver-client-sse-spring:3.2.2"

application.yaml

optim-server:
  sse:
    client:
      master:
        url: https://master-host/

@SpringBootApplication
@Import(SseSpringConfig.class)
public class ApiSSEClientApplication {
    public static void main(String[] args) {
        SpringApplication.run(ApiSSEClientApplication.class, args);
    }
}

import com.decisionbrain.optimserver.client.sse.SseListener;

@Service
public class MyService
{
    private final SseListener sseListener;
    
    void listenToOptimServerJobStatusEvents() {
        String jobId = "created job Id";
        sseListener.register(jobId, new JobEventListener() {
            @Override
            public Consumer<JobStatusEvent> onJobStatusEvent() {
                return (jobStatusEvent) -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()));
            }
        });
    }    
}

3.3.0:

implementation "com.decisionbrain:spring-boot-starter-optimserver-sse-client:3.3.0"

application.yaml

optim-server:
  url: https://master-host/

@SpringBootApplication
@EnableOptimServerClient
public class ApiSSEClientApplication {
    public static void main(String[] args) {
        SpringApplication.run(ApiSSEClientApplication.class, args);
    }
}

import com.decisionbrain.optimserver.client.java.async.api.JobExecutionAsyncApi;
import com.decisionbrain.optimserver.client.java.async.api.JobEventSource;

@Service
public class MyService
{
    private final JobExecutionAsyncApi jobExecutionAsyncApi;
    
    public MyService(JobExecutionAsyncApi jobExecutionAsyncApi) {
        this.jobExecutionAsyncApi = jobExecutionAsyncApi;
    }
    
    void listenToOptimServerJobStatusEvents() {
        String jobId = "created job Id";
        JobSubscriptionFilter filter = new JobSubscriptionFilter(jobId);
        JobEventSource eventSource = jobExecutionAsyncApi.getJobEventSource(filter);

        eventSource.statusEvents()
                .subscribe(new ConsumerSubscriber<>(
                        jobStatusEvent -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()))
                ));

        eventSource.connect();
    }    
}

The Jersey SSE Asynchronous API client

The Asynchronous API has been fully rewritten in Optimization Server 3.3.0.

3.2.2:

implementation 'com.decisionbrain:optimserver-client-sse-jersey:3.2.2'

public class SampleJavaSSEClient {
    private static SseListener createSseClient() {

        // This is the main keycloak configuration to authenticate on your Master API.
        final KeycloakClientConfig keycloakClientConfig = KeycloakClientConfig.builder()
                .url(KEYCLOAK_URL)
                .realm(KEYCLOAK_REALM)
                .client(KEYCLOAK_CLIENT)
                .user(KEYCLOAK_USER)
                .password(KEYCLOAK_PASSWORD)
                .build();
        final AuthenticationService authenticationService = new KeycloakAuthenticationServiceImpl(keycloakClientConfig);

        // Configure the master API url and optionally the SSE reconnect time.
        final SseJerseyConfig sseJerseyConfig = SseJerseyConfig.builder()
                .apiUrl("http://" + DBOS_MASTER_HOST + ":" + DBOS_MASTER_PORT)
                .reconnect(2000)
                .build();

        // Create the listener service instance
        return new JerseySseListenerImpl(authenticationService, sseJerseyConfig);
    }

    public static void main() {
        try {
            String jobId = "created job Id";
            final SseListener seeClient = createSseClient();
            seeClient.register(jobDefinition.getId(), new JobEventListener() {
                @Override
                public Consumer<JobStatusEvent> onJobStatusEvent() {
                    return (jobStatusEvent) -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()));
                }

                @Override
                public Consumer<Throwable> onJobStatusEventError() {
                    return (throwable) -> LOGGER.error("Exception while listening to the job status", throwable);
                }
            });
            jobExecutionClient.startAsyncJobExecution(jobId, null);
            LOGGER.info("Waiting for a solution...");
        } catch (ApiException e) {
            logApiException(e);
        } catch (Exception e) {
            LOGGER.error("An error occurred: ", e);
        }
    }
}

3.3.0

implemmentation 'com.decisionbrain:optimserver-sse-client:3.3.0'

public class SampleJavaSSEClient {
    
    private static JobExecutionAsyncApi createSseClient() {

        HttpHeadersProvider keycloakHeaders = new KeycloakHttpHeadersProvider(
                KEYCLOAK_URL,
                KEYCLOAK_REALM,
                KEYCLOAK_CLIENT,
                KEYCLOAK_USER,
                KEYCLOAK_PASSWORD
        );

        ApiClientConfiguration apiConf = ApiClientConfiguration.builder()
                .baseUri(new URI("http://" + DBOS_MASTER_HOST + ":" + DBOS_MASTER_PORT).normalize())
                .requestHeaders(keycloakHeaders)
                .build();

        return new SseJobExecutionAsyncImpl(
                new SseEventSourceFactory(apiConf),
                new ObjectMapper()
        );
    }
    
    public static void main() {
        try {
            String jobId = "created job Id";
            final JobExecutionAsyncApi jobExecutionAsyncApi = createSseClient();
            JobSubscriptionFilter filter = new JobSubscriptionFilter(jobId);
            
            JobEventSource eventSource = jobExecutionAsyncApi.getJobEventSource(filter);

            eventSource.statusEvents()
                    .subscribe(new ConsumerSubscriber<>(
                            jobStatusEvent -> LOGGER.info(String.format("[STATUS]: %s", jobStatusEvent.getStatus().name()))
                    ));

            eventSource.connect();
        } catch (ApiException e) {
            logApiException(e);
        } catch (Exception e) {
            LOGGER.error("An error occurred: ", e);
        }
    }
}

Worker (Java)

Break changes: adjust code

The ‘File’ type

The File ExecutionContext::getBucketFile method has moved to InputStream ExecutionContext::getBucketFile.

The changes to integrate are the same as for

described above

ExecutionContext::startJobAndWaitForCompletion

The signature has changed:

- Optional<SolutionDTO> startJobAndWaitForCompletion(JobSubmitRequestDTO jobSubmitRequestDTO)
+ Optional<JobSolution> startJobAndWaitForCompletion(JobSubmitRequestDTO jobSubmitRequestDTO)

The path to get the bucket reference of the solution moves from:

SolutionDTO.getSolution.getBucketValue("output-key").getBucketId

to:

JobSolution.getOutputs[name="output-key"].getBucketId
Deprecations

Since the worker library relies on the Master API Spring Boot starter it must comply with its configuration.

The properties in the application.yaml file:

  • master.url
  • master.jwtKey

have moved to:

  • optim-server.url
  • optim-server.jwt.jwtKey

The former values are still taken into account to ensure backward compatibility.

The properties are migrated by calling the Python helper script :

python -m optimserver.workerapp.migration.migrate_workerapp_config --src=3.2.2 --dst=3.3.0 --type=APPLICATION_YAML --input INPUT --output OUTPUT
Make sure the output file complies with the description below.

3.2.2:

master:
  url: https://master-host/
  jwtKey: ...

3.3.0:

optim-server:
  url: https://master-host/
  jwt:
    jwtKey: ...

Worker (Python).

Deprecations

The worker shell relies on the Master API Spring Boot starter as well.

The properties in the application.yaml file:

  • master.url
  • master.jwtKey

have moved to:

  • optim-server.url
  • optim-server.jwt.jwtKey

The former values are still taken into account to ensure backward compatibility.

The properties are migrated by calling the Python helper script :

python -m optimserver.workerapp.migration.migrate_workerapp_config --src=3.2.2 --dst=3.3.0 --type=APPLICATION_YAML --input INPUT --output OUTPUT
Make sure the output file complies with the description below.

3.2.2:

master:
  url: https://master-host/
  jwtKey: ...

3.3.0:

optim-server:
  url: https://master-host/
  jwt:
    jwtKey: ...

cplex and wod charts

Deprecations

The environment variables in the values.yaml file

  • MASTER_URL
  • MASTER_JWTKEY

have moved to:

  • OPTIMSERVER_URL
  • OPTIMSERVER_JWT_JWTKEY

The properties are migrated by calling the Python helper script :

python -m helm.migration.migrate_helm_values --src=3.2.2 --dst=3.3.0 --chart=DBOS_WORKER --input INPUT --output OUTPUT

Make sure the output file complies with the description of the section below.

3.2.2:

env:
  - name: MASTER_URL
    value: ...
  - name: MASTER_JWTKEY
    valueFrom: ...

3.3.0:

env:
  - name: OPTIMSERVER_URL
    value: ...
  - name: OPTIMSERVER_JWT_JWTKEY
    valueFrom: ...

The script changes the names of the variables, so it can be applied to any values.yaml file that describes a worker deployment