The decision is based on an examination appeal and relates to an adaptive deep learning models calibrated and personalized for (“users” of) autonomous vehicles. The Applicant argued that continuous learning of a deep learning model and it allowed for efficient fine-tuning through fast re-training with minimal data.
The Board disagreed as continuous learning aspect is not part of the claimed invention, and person skilled in the art implements a method for providing a deep learning model to an autonomous vehicle which uses a static library of deep learning models for different vehicles, from which a managing device selects a model corresponding to the vehicle in question. Thus, the application was refused.
Here are the practical takeaways from the decision: T 2412/22 (Adaptive driving models/STRADVISION) of November 24, 2024, of the Technical Board of Appeal 3.05.06
Key takeaways
The question is not what the prior art discloses, but how the person skilled in the art would modify it, e.g. in order to improve it”.
In general, the person skilled in the art would consider well-known alternatives, this applies to adapting the model on a central device and sending the updated model to the vehicle
The number of differences over a certain piece of prior art is neither decisive nor a reliable indicator for the presence of an inventive step. First, the number of differences itself may be, and often is, deceiving. One modification may imply or make obvious several other differences. Secondly, whether several modifications combine to provide an inventive overall contribution does not depend on their number. For instance they may be obvious solutions to independent, “partial problems”.
The invention
The Board defined the invention as follows:
1. The application relates to the provision of adaptive deep learning models calibrated and personalized for (“users” of) autonomous vehicles.
1.1 According to the application, known autonomous vehicles use “legacy” deep learning models trained by using data collected per country or region, and this is not satisfactory for drivers with different “tendencies”, which the board understands to refer to driving behaviours. The application therefore proposes that the learning models be customized (see page 1).
1.2 In order to do this, the system maintains a collection of legacy models. For any individual vehicle, a suitable legacy model is selected for fine tuning. The legacy model is selected to have been trained for “video” conditions (i.e. vehicle type, place, time, or driver) similar to those applying to the vehicle of interest. The tuning is realised by further training the legacy models with data collected from the vehicle of the specific user (see page 2). This data is labelled by a combination of automatic and manual labelling.
-
Claim 1 of Main Request
Is it patentable?
The Applicant and the Board agreed that the differences were as follows:
3. In its communication the Board identified the following set of differences between claim 1 and D1:
(a) the fine tuning taking place on a server (“managing device”) rather than on the vehicle, and the subsequent transmission of the customised model to the vehicle
(b) the existence of a library of legacy models for specific vehicles
(c) the selection of one model for updating based on a relationship score determined using video data information
(d) a certain data labelling scheme, as recited in the penultimate claim paragraph.
And the Applicant argued that features a to c were technical and inventive for the following reasons:
6.1 The invention related to continuous learning of a deep learning model for a specific autonomous vehicle. The model was retrained with specific video data for specific circumstances and stored in a library containing the various models. The storage of models retrained for various circumstances allowed for efficient fine-tuning through fast re-training with minimal data. The selection step based on video data information ensured that the proper model was selected and updated.
…
6.2.2 The skilled person also had no reason to perform the model adaptation on a server. There was enough computing power on a vehicle to perform re-training, and the need to communicate with a server might compromise real-time adaptation. In fact, the real-time requirement of D1 taught away from a centralized solution. Sending video data, waiting for computation and receiving the adapted model caused time delays which did not allow real-time adaptation.
But the Board disagreed:
7. The Board remarks first that the Appellant’s conceptual presentation of the invention (see 6 and 6.1 above) does not entirely correspond to the claimed invention, which is less detailed and therefore of broader scope.
7.1 In particular, the continuous learning aspect is not part of the claimed invention. The library is not defined to be dynamic in content, because the claim does not specify a storage of the updated model in the library.
…
7.3 The claim therefore covers a method for providing a deep learning model to an autonomous vehicle based on a static library of deep learning models for different vehicles, from which a managing device selects a model corresponding to the vehicle in question, retrains it using the vehicle video data, and transmits it to the vehicle.
The Applicant argued that library model were not needed in D1, but the Board again disagreed:
9. Thereby the person skilled in the art implements a method for providing a deep learning model to an autonomous vehicle which uses a static library of deep learning models for different vehicles, from which a managing device selects a model corresponding to the vehicle in question.
The Board addressed why the features were obvious:
10.1 However, for inventive step, the question is not what D1 discloses, but how the person skilled in the art would modify it, e.g. in order to improve it.
10.2 In general, the person skilled in the art would consider well-known alternatives. In the case in hand, this applies to adapting the model on a central device and sending the updated model to the vehicle. There are, in the Board’s view, good reasons for doing this, in particular the fact that more computational resources may be – and generally are – available on the server, and that this way the on-board computer, with necessarily limited resources, is free to perform other tasks.
10.3 Indeed this requires data transmission, but the trade-off is known to the person skilled in the art, who would choose one of the two options depending on the circumstances.
Finally, the applicant argued that D1 focused on real-time adaptation during the operation of the vehicle and that this taught away from a centralized solution. However, the Board disagreed as D1 also conserned real-time adaptation, and it was know that certalised computation may offer the same level of real time as adaptation in the device.
Auxiliary Request
In the first and fourth requests it is defined that “the deep learning models have the video data information as their tagged data”, which according to the Appellant this simplifies and speeds up the model selection step. However, this did not change the assessment of the Board.According to the Appellant this simplifies and speeds up the model selection step.
17. The Board remarks that once a library of models for various types of vehicles is defined (see point 9 above), the models need to be indexed by type, so that they can be differentiated and retrieved. This indexing implies a “tag” of some form as claimed.
Claims 1 of auxiliary requests 2, 3 and 4′ (also) define a cross-validation step between the manual and the automatic labeling. The Board considered they were obvious and noted as follows:
20. However, if this were the case, it would also imply that the cross-validation would be obvious for the skilled person: Given that D1 already specifies a combination of automatic and manual labeling (paragraph 78, see 2.1 above), the person skilled in the art, knowing how to improve accuracy by cross-validation between automatic and manual labeling, would use cross-validation without exercising any inventive skill.
Finally, the Applicant argued that the invention had number of differences and starting from D1 requred a number of modification. But the Board did not accept this argument:
22. The Board remarks that the number of differences over a certain piece of prior art is neither decisive nor a reliable indicator for the presence of an inventive step.
22.1 First, the number of differences itself may be, and often is, deceiving. One modification may imply or make obvious several other differences. For instance, as in the case in hand, performing the computations on a server instead of on the user vehicle, implies data transmission, and with it a host of other associated “differences” which may or may not be specified in a claim, like an antenna, a transmission protocol etc. A library implies storage, indexing, a retrieving mechanism and so forth. Also, in complex systems it is very easy to accumulate a large number of individual differences while simply considering the different options available to the person skilled in the art.
22.2 Secondly, whether several modifications combine to provide an inventive overall contribution does not depend on their number. For instance they may be obvious solutions to independent, “partial problems”.
Therefore, the Board decided that the appeal was dismissed, and the patent application was refused.
More information
You can read the full decision here: T 2412/22 (Adaptive driving models/STRADVISION) of November 24, 2024, of the Technical Board of Appeal 3.05.06