Author Archive

Distribution of electronic assets with reduced load to the server: technical

The application relates to a method for the distribution of electronic assets to a test application in a manufacturing process. The Board noted that the solution of providing a daemon that takes over the management of assets reduces the load of the server and thus solves the technical problem of how to minimise the overhead of the server. Since none of the cited documents would lead the skilled person to the solution without hindsight, the subject-matter was inventive. Here are the practical takeaways from the decision T 0447/19 of May 11, 2022, of the Technical Board of Appeal 3.4.03:

Key takeaways

Reducing the load of the server is a technical effect, and it solves a technical problem of how to minimise the overhead of the server

The invention

Products today are on a System-on-Chip (SoC) where manufacturers may use the same SoC with various features enabled/disabled in order to differentiate the final products in the market. In distributed manufacturing, the devices are manufactured at remote locations, and unauthorized enablement of features represents significant revenue loss to companies. Therefore, there is a need for the central producer to monitor and control the device to avoid fraud by local manufacturing (sub)contractors.

Common practice is to provide cryptographic keys with each electronic asset which is distributed from the central producer to a local server at each remote manufacturer. The assets are then tested by an agent (log generated) before inserting into the device.

The invention relates to the management of the asset distribution where the agent is executed as a separate daemon process, independent from the corresponding test application. This allows the agent takes over the communication with the appliance and the request/receipt of electronic assets for the corresponding test application.

Therefore, it is not necessary to establish a connection with the test application or monitor the usage of the assets every time the application needs assets for insertion into the device, as the daemon takes over this task. Moreover, assets left at the daemon’s cache at the termination of the test application are not lost but can be used by a subsequent instance of the test application

Claim 1 (Sole Request)

A method of controlling the distribution of electronic assets to a test application (116b) in a manufacturing process, said method comprising:

providing a daemon application programming interface (API) (23) on an instance of said test application to provide assets upon detecting a request therefor, and to obtain log data from said instance of said test application during testing;

initiating a daemon (25) in connection with said daemon API to obtain said log data from said daemon API (23) and to provide said assets to said daemon API, said daemon hosting an agent API (21) for communicating with an appliance (18) remote to said instance of said test application;

utilizing said agent API to obtain a batch comprising a plurality of assets and to provide one or more log reports containing said log data separately from said instance of said test application; and

said daemon: caching said assets to provide a quantity of said assets to said daemon API upon request therefrom to enable said daemon API to provide said assets to said instance of said test application thereby avoiding session establishment between said instance of said test application and said appliance for obtaining said electronic assets; and maintaining a record of assets not used when said instance of said test application terminates for a next instance of the test application.

Is it patentable?

The Examining Division refused the application and considered the subject-matter of the claims lacked inventive step over documents D1 (WO 2006/133545 A1) combined with document D3 (US 2008/0040550 A1). The distinguishing features and the technical problem were defined as follows:

3.1 It is common ground that document D1 represents the closest prior art.

According to the appellant, claim 1 differs from D1 in that (numbering by the board in line with the impugned decision):

(a) the agent software program is executed as a daemon process separately from the test application and comprises an agent API for communicating with the appliance, and

(b) the daemon maintains a record of assets not used when the instance of the test application terminates for a next instance of the test application.

These distinguishing features were also identified by the examining division, although the division interpreted feature (b) differently, in that it was not the daemon which maintained the assets for the next instance of the test application but simply “assets not used … are maintained …” (see the middle of page 6 of the impugned decision). Since the board agrees with the appellant’s interpretation of feature (b), it also adopted the appellant’s definition of the distinguishing features (see page 3 of the statement of the grounds of the appeal).

3.2 According to the appellant, these two distinguishing features combine to provide the technical effect of the daemon taking over the management of assets, reducing thus the load of the server. Hence, the objective technical problem the skilled person starting from D1 would be faced with was how to minimise the overhead of the server 18 when dealing with equipment 20 (ibid., page 5).

3.2.1 As the appellant further explained during the oral proceedings, the daemon was monitoring the use of the cached assets by the test application. Receiving the log data from the test application regarding the assets inserted into the devices, the daemon was monitoring the use of the cached assets (see also Figure 6B of the published application). In D1 it was the controller of the producer which was receiving all the log data from the key agent and was monitoring which keys (assets) were inserted into the devices, whether there were any unused keys left, etc. (see e.g. Figures 1 and 4 of D1). Hence, by having this monitoring performed by the daemon, the overhead of the server was indeed minimised.

3.3 The board accepts the appellant’s formulation of the objective technical problem.

In the decision under appeal, the examining division considered that the two identified distinguishing features did not combine to produce a synergistic technical effect and assessed them separately. However, since the board does not accept the division’s interpretation of the second distinguishing feature, it does not follow its formulation of the technical problem, either. Since both distinguishing features (a) and (b) relate to the daemon, and the appellant’s formulation of the objective technical problem is considered plausible, the board decided to follow the appellant in this respect.

The Board did not agree with the Examination Division that the skilled person will arrive at the feature from document D3 as it provides a different solution to the problem:

3.4 The examining division referred also to D3. D3 describes a network in which a series of client applications 120 are connected to a directory server 110 (see Figure 1). Normally such client applications access the directory server by establishing a direct connection through a binding operation, which initiates a protocol session between the application and the server, allows authentication of the client to the sever, etc. (see paragraph [0009]).

A problem in such a network architecture is that each application needs to establish a direct connection to the server before it can request any information from it. The server needs to establish separate connections (sessions) to each application for any exchange of information to take place. This increases the load on the server affecting its performance (see paragraph [0010]).

D3 solves this problem by installing a Light Directory Access Protocol (LDAP) caching daemon 210 between the applications and the server (see Figure 2). The daemon obtains data from the server and stores (caches) them in its data cache so that it can directly provide them to the requesting applications without any need for the applications to connect to the server. At the same time, the daemon connects to the directory server and retrieves any information requested by an application but not stored in its cache. In this way, the server has to manage only one individual connection (to the daemon) and can perform its main task of information retrieval more efficiently (see paragraphs [0023] to [0027]).

Furthermore, the Board also agreed with the applicant that even starting from document D1, the skilled person will seek other solutions to the problem, and will not arrive at the solution without hindsight:

3.7 The examining division considered that executing the key agent (21) as a separate background daemon process would have been an obvious choice for the skilled person which would be aware of the advantages of such a modular implementation based only on their common general knowledge. Moreover, such an a implementation would also have been obvious in view of the teaching of D3 (see first two paragraphs on page 7 of the impugned decision).

3.7.1 As explained previously, the board does not agree with the examining division’s separate assessment of the distinguishing features or its formulation of the objective technical problem.

Even if this argument were followed and it were to be accepted that moving the key agent (21) of the manufacturing equipment (see Figure 1 of D1) to a separate daemon process would have been obvious to the skilled person, the board notes that there would still be feature (b) missing from such an implementation to arrive at the claimed invention.

In the claimed invention it is the daemon which, using the log data received from the test application, maintains a record of the unused assets remaining in its cache. The key agent (21) in D1 does not do anything similar. As explained in point 3.6 above, the monitoring of the keys (assets) and their usage in D1 is performed by the controller of the producer and not by the key agent. Implementing the key agent in a separate daemon process would not change this. The log data would still be sent by the manufacturing equipment (20) to the server (18) through the key agent (21). The evaluation and monitoring of the key usage would still be done by the controller of the producer based on the received log report “R”.

Since according to the method of D1 the evaluation and monitoring of key usage based on the log data are effected at the producer and not at the server of the manufacturer, the skilled person wishing to reduce the load to the server would not have any reason to move this functionality from the producer to the key agent. They would rather seek functionalities that could be moved from the server of the manufacturer to the key agent.

Hence, there is no incentive in D1 for the skilled person to introduce any key usage monitoring functionality at the key agent without hindsight. D3, which does not mention any keys/assets or any monitoring of the usage/access of the data cached at the daemon by the applications would not provide any such incentive, either.

3.8 The board’s conclusion is, therefore, that the subject-matter of claim 1 involves an inventive step within the meaning of Article 56 EPC.

Therefore, the Board concluded that the subject-matter of the claims involves an inventive step, and the appeal is allowed with an order to grant a patent

More information

You can read the whole decision here: T 0447/19 of May 11, 2022, of the Technical Board of Appeal 3.4.03

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

Privacy policy Yes, I consent to the collection, processing and use of my above-mentioned personal data for the purposes of processing my message and for the purposes of contacting me via email. The legal basis of the processing shall be formed by my consent pursuant to Art. 6 (1) lit. a GDPR. The data will be deleted three months after expiry of the purpose, provided that longer retention periods are not required by law. I can revoke this consent with future effect at any time. I have taken note of the privacy statement and consent to it. With regard to the processing of my data, I am entitled to inalienable rights, information on which can be found in the privacy statement.

* = Required field

Preston Richard

Preston is a European patent attorney at BARDEHLE PAGENBERG. He handles patent applications in the field of electronics and communication technology, particularly hardware and software inventions related to wireless communication and entertainment devices.

www.bardehle.com/de/team/richard-preston

Reliable measure of the overall blood glucose variability: technical

The application relates to a system for measuring blood glucose variability, by acquiring a plurality of blood glucose data points over a predetermined time period. The Board noted that the technical effect was achieved only for longer periods of observation. Since the claim did not specify the period, the technical effect was not achieved over the whole scope of the claim. However, the amended claim with the limitation of the specific period (7 days) rendered the distinguishing features technical and thus was considered for the assessment of the inventive step, and found to be inventive. Here are the practical takeaways from the decision T 2681/16 of March 14, 2022 of the Technical Board of Appeal 3.2.02:

Key takeaways

Features that relate solely to the algorithm used in the claimed system, when taken in isolation, are non-technical. As such, they can support the presence of an inventive step only if they credibly contribute to producing a technical effect serving a technical purpose

It is irrelevant if features might contribute to the technical character of the claimed subject-matter and thus to an inventive step for other parts of the claimed subject-matter. A prerequisite for meeting the requirement that the claimed invention is inventive over the whole scope of the claim is indeed that it is also technical over the whole scope.

The invention

Blood glucose variability is an important parameter in diabetes management, and monitoring this parameter may help patients to improve their behaviour and self-treatment practices to reduce the risk.

For this purpose, the patent application provides a system for measuring blood glucose variability, by acquiring a plurality of blood glucose data points over a predetermined time period, for example over a number M of days; a processor programmed to calculate a risk index based on the acquired blood glucose data points and to classify it into a risk category, and a display module to display the determined risk category. For this, an algorithm to calculate the risk index from the acquired blood glucose data points is presented.

The applicant also provided experimental validation studies in the description confirming, that when calculated from a sufficiently large set of blood glucose data points spanning a sufficiently long period of time, the risk index turns out to be equally predictive. Therefore, the method was a reliable measure of the overall blood glucose variability compared to other measures of variability previously used.

Claim 1 (Main Request)

A system adapted to measure blood glucose variability, said system comprising:

an acquisition module for acquiring a plurality of blood glucose data points;

a processor programmed to:

transform the plurality of blood glucose data points from a blood glucose range to a transformed range according to a transforming function;

calculate a risk value for each of the transformed plurality of blood glucose data points; characterised by the processor being further programmed to:

calculate a plurality of risk range values, each based on a maximal risk value of at least two of the calculated risk values within a period of time with a predetermined duration;

calculate at least one composite risk range value based on a summation of a plurality of the calculated risk range values; and

define risk categories using cutoff points of composite risk range values and classifying the calculated composite risk range value into one of the risk categories; and

a display module adapted to display the risk category that the calculated composite risk range value is classified into.

Claim 1 (Second Auxiliary Request)

A system adapted to measure blood glucose variability, said system comprising:

an acquisition module for acquiring a plurality of self-monitored blood glucose data points, wherein the plurality of self-monitored blood glucose data points span a period of at least one day;

a processor programmed to:

transform the plurality of blood glucose data points from a blood glucose range to a transformed range according to a transforming function, wherein the minimal and maximal values of the transformed range are -SQRT10 and SQRT10, respectively, wherein the transforming function is:

where BG is a blood glucose value, (alpha, beta, gamma) = (1.026, 1.861, 1.794) if BG is measured in mM, and (alpha, beta, gamma) = (1.084, 5.381, 1.509) if BG is measured in mg/dl; calculate a risk value for each of the transformed plurality of blood glucose data points by:

defining a BG risk space, wherein the BG risk space is:

defining a left branch of the BG risk space representing a risk of hypoglycemia as:

rl(BG) = r(BG) if f(BG) < 0 and 0 otherwise;

defining a right branch of the BG risk space representing a risk of hyperglycemia as:

rh(BG) = r(BG) if f(BG) > 0 and 0 otherwise; characterised by the processor being further programmed to:

calculate a maximal hypoglycemic risk value for the plurality of self-monitored blood glucose data points for each day as:

where ni is the number of readings for each day i and x**(i)1, x**(i)2, …, x**(i)ni are the ni self-monitored blood glucose data points for each day i;

calculate a maximal hyperglycemic risk value for the plurality of self-monitored blood

glucose data points for each day i as:

where ni is the number of readings for each day i and x**(i)1, x**(i)2, …, x**(i)ni are the ni self-monitored blood glucose data points for each day i;

calculate average daily risk range as:

where the plurality of blood glucose data points are collected on days i = 1, 2, … , M; and

define risk categories using cutoff points of average daily risk range values and classifying the calculated average daily risk range into one of the risk categories;

a display module adapted to display the risk category that the calculated average daily risk range is classified into.

Is it patentable?

The Examining Division refused the application and considered the subject-matter of the claims lacked inventive steps starting from D5 (B. P. Kovatchev et al., “Quantifying Temporal Glucose Variability in Diabetes via Continuous Glucose Monitoring: Mathematical Methods and Clinical Application”, Diabetes Technology & Therapeutics, vol. 7, no. 6, 2005).

In the Appeal proceedings, the Board raised new objections to the lack of clarity and support in the description (Art.84 EPC). The Board indicated that although the claims included a predetermined time as a day, the description consistently discloses that the risk index must be calculated over a period of time. The applicant filed amended claims (Second to fourth Auxiliary Request) including features which overcame this objection.

The Board then considered if the Second Auxiliary Request was inventive over the cited document D5, which discloses a statistical risk analysis to quantify the blood glucose variability of a diabetic patient from self-monitored blood glucose data points.

The appellant argued that the claim was inventive in that the way the acquired blood glucose data points were processed to derive a measure of the blood glucose variability (features (a) – (c) ) and how the average daily risk range was calculated.

The Board did not agree with the argument:

3.2.3 As acknowledged by the appellant, features (a)-(c) relate solely to the algorithm used in the claimed system to process the acquired blood glucose data points. These features, when taken in isolation, are non-technical. As such, they can support the presence of an inventive step only if they credibly contribute to producing a technical effect serving a technical purpose (Case Law of the Boards of Appeal, 9th edition, 2019, I.D.9.1.3 c)).

3.2.4 The appellant submitted that these features had the technical effect of providing an overall measure of the glucose variability (i.e. equally sensitive to both hypo- and hyperglycemic events) and a prediction of glycemic events that were better than, or at least alternative to, those used in D5.

3.2.5 While this effect does appear to be achieved for longer periods of observation, as in the experimental validation studies reported in the description, the Board does not find it credible that it is achieved if a very small number of blood glucose data points is used to calculate ADRR, in particular when the minimum number of blood glucose data points required by claim 1 is used, i.e. two data points spanning a period of one day only; see the discussion in point 2.1 above, especially the last paragraph. The appellant did not provide any convincing arguments demonstrating that this was the case.

3.2.6 It follows that, at least for these parts of the claimed subject-matter, features (a)-(c) do not contribute to the technical character of the claimed subject-matter and therefore cannot support the presence of an inventive step.

Contrary to the appellant’s contention, it is irrelevant that features (a)-(c) might contribute to the technical character of the claimed subject-matter and thus to an inventive step for other parts of the claimed subject-matter, in particular when more blood glucose data points are used. A prerequisite for meeting the requirement that the claimed invention is inventive over the whole scope of the claim is indeed that it is also technical over the whole scope. The requirement has not been met, as features (a)-(c) contribute to the technical character only for certain specific embodiments of the claimed invention (see G 1/19, Reasons 84).

The fact that claim 1 defines a system and not a method, as put forward by the appellant, is immaterial. Indeed, claim 1 does not merely require that the system be suitable for calculating ADRR from a higher number of blood glucose data points. Rather, the minimum number of blood glucose data points used to calculate ADRR is a feature of the algorithm specifically programmed in the processor of the claimed system. It is also a feature of the acquisition module, which must be specifically adapted to acquire at least this number of blood glucose data points. Thus, this minimum number is a feature of the claimed system itself.

Since the claim was not technical over the whole scope (i.e., the claimed technical effect of the overall measure of the glucose variability and a more reliable risk prediction was not credible for a shorter period of observation), the Board considered the claim lacked inventive step.

The applicant then amended the claim and filed a fourth Auxiliary Request by changing the expression “at least one day” to “at least seven days” and using at least three blood glucose data point readings per day.

Claim 1 (Fourth Auxiliary Request)

A system adapted to measure blood glucose variability, said system comprising:

an acquisition module for acquiring a plurality of self-monitored blood glucose data points, wherein the plurality of self-monitored blood glucose data points span a period of at least seven days;

a processor programmed to:

transform the plurality of blood glucose data points from a blood glucose range to a transformed range according to a transforming function, wherein the minimal and maximal values of the transformed range are -SQRT10 and SQRT10, respectively, wherein the transforming function is:

where BG is a blood glucose value, (alpha, beta, gamma) = (1.026, 1.861, 1.794) if BG is measured in mM, and (alpha, beta, gamma) = (1.084, 5.381, 1.509) if BG is measured in mg/dl; calculate a risk value for each of the transformed plurality of blood glucose data points by:

defining a BG risk space, wherein the BG risk space is:

defining a left branch of the BG risk space representing a risk of hypoglycemia as:

rl(BG) = r(BG) if f(BG) < 0 and 0 otherwise;

defining a right branch of the BG risk space representing a risk of hyperglycemia as:

rh(BG) = r(BG) if f(BG) > 0 and 0 otherwise; characterised by the processor being further programmed to:

calculate a maximal hypoglycemic risk value for the plurality of self-monitored blood glucose data points for each day as:

where ni is the number of readings for each day i and x**(i)1, x**(i)2, …, x**(i)ni are the ni self-monitored blood glucose data points for each day i;

calculate a maximal hyperglycemic risk value for the plurality of self-monitored blood

glucose data points for each day i as:

where ni is the number of readings for each day i and x**(i)1, x**(i)2, …, x**(i)ni are the ni self-monitored blood glucose data points for each day i;

calculate average daily risk range as:

where the plurality of blood glucose data points are collected on days i = 1, 2, … , M; and

define risk categories using cutoff points of average daily risk range values and classifying the calculated average daily risk range into one of the risk categories;

a display module adapted to display the risk category that the calculated average daily risk range is classified into;

wherein the plurality of blood glucose data points includes at least three blood glucose data point readings per day.”

The Board noted that these features were not obvious for the following reasons:

6.2.1 Claim 1 of the fourth auxiliary request requires a minimum of 3 x 7 = 21 data points spanning a period of seven days. Even though this corresponds to a smaller set of data points than used in the experimental validations studies reported in the description (see point 2.1 above), the Board finds it credible that this number of blood glucose data points and the period they span still represent a sufficiently large sample to be statistically significant. The Board is therefore satisfied that, in these conditions, the technical effect put forward by the appellant (point 3.2.4 above) is achieved over the whole claimed subject-matter. Accordingly, features (a)-(c) can a priori support the presence of an inventive step.

6.2.2 The Board concurs with the appellant’s view that, contrary to the Examining Division’s view (point 15.4.1 of the decision under appeal), the person skilled in the art, starting from D5, would have had no motivation to modify the definition of LBGI and HBGI given in D5 so as to determine maximum risk values instead of average risk values (feature (a)).

Indeed, as disclosed in D5, page 852, top of the left-hand column, the risk indices LBGI and HBGI have been specifically constructed so as to be a “measure of the frequency and extent” of the low and high blood glucose readings, respectively. Replacing the average risk value averaged over each period of time by the maximum value reached over said period would erase information on the frequency and extent of the blood glucose excursions occurring during that period of time, as argued by the appellant. This would be contrary to the purpose of the risk analysis presented in D5. Without the benefit of hindsight, the person skilled in the art would have therefore not envisaged such a modification.

Moreover, none of the other documents D1-D4 and D6 cited in the decision under appeal discloses or suggests feature (a).

At least for this reason, the subject-matter of claim 1 of the fourth auxiliary request involves an inventive step over D5, contrary to the Examining Division’s finding.

Therefore, the Board concluded that the subject-matter of the claim of fourth Auxiliary Request involves an inventive step, and the appeal is allowed with an order to grant a patent

More information

You can read the whole decision here: T 2681/16 of March 14, 2022 of the Technical Board of Appeal 3.2.02

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

www.bardehle.com/de/team/richard-preston

Eliminating spurious or unstable candidates: technical

The application relates to identifying so-called keypoints in a digital image, for e.g. scene or object identification/matching. The Board agreed with the applicant that the distinguishing feature provides the effect of eliminating spurious or unstable candidate keypoints since the points outside the validity range of the approximation are eliminated. Here are the practical takeaways from the decision T 1482/19 (Scale space approximation/TELECOM ITALIA) of February 11, 2022, of the Technical Board of Appeal 3.5.06:

Key takeaways

Eliminating spurious or unstable candidate keypoints, due to the effects of the approximation, is a technical problem

The invention

The application relates to a computation method for identifying so-called keypoints in a digital image, for e.g. scene or object identification/matching.

The physical parameters of an image such as luminance are used to identify the size and position of the salient features of the image, and its location by identifying the keypoint, which is the centre of the detail. The known method detects keypoints as extrema in the image “scale space”. However, these are not reliable as they must settle for a solution that trades off efficiency with completeness.

The application proposes a method of approximating the scale space to determine keypoint. Although scale space approximation was a generally known computational method, it was not known to be used with keypoints and the invention additionally included eliminating spurious or unstable candidate keypoints.

Claim 1

A method for identifying keypoints in a digital image (I) comprising a set of pixels, each pixel having associated thereto a respective value of an image representative parameter, said method comprising:

– approximating (108), without explicitly computing, a filtered image (L(σ), 102) of said digital image (I), said filtered image being obtained from said digital image (I) by means of a filtering function, said filtering function being a Laplacian of Gaussian or a Difference of Gaussians and having a filtering parameter (σ) being the standard deviation of the Gaussian function, said approximating comprising:

a) generating (102) a set of base filtered images (L(σ)) of said digital image (I), each base filtered image (L(σi) being the digital image (I) filtered by means of a respective base filter, each base filter being a matrix the elements of which are obtained by calculating said filtering function with a respective value (σi) of the filtering parameter;

b) for each pixel (xt,yt) of at least a subset of said set of pixels of the digital image (I), approximating (106) the filtered image (L(xt, yt, σ)) obtained from said digital image (I) by means of said filtering function having a filtering parameter value by means of a respective approximation function based on the base filtered images, said approximation function being a function of the filtering parameter (σ) within a predefined range (104) of values of the filtering parameter (σ), said predefined range ranging from a lowest value of the filtering parameter to a highest value of the filtering parameter, wherein:

– wherein said approximating function is a polynomial expressed as follows:

wherein

L(x, y, σ) is the filtered image to be approximated

c(x, y) is the polynomial coefficient for σ**(r)

wherein said polynomial coefficients cr(x, y) are obtained as a linear combination of said base filtered images according to the following equations:

wherein F**(T) is calculated by solving the following equation:

SF=W**(T)

wherein:

F is the unknown which, after being calculated and transposed, gives F**(T)

S is a matrix formed by specific values of σi as follows:

wherein σ’i (with i varying from 1 to q) are specific values of the filtering parameter σ

W is a matrix calculated by solving the following equation:

AW = D

wherein

A is a matrix formed by the base filters, wherein matrix A has a number of columns equal to the number of base filters, wherein each column of matrix A represents a respective base filter, wherein each column of matrix A is built by arranging in columns the columns of each base filter

D is a matrix each element of which is obtained by calculating the filtering function, for each point (x, y) and for each of said specific values σ’j (with j varying from 1 to q)

W is the unknown which, after being calculated and transposed, gives W**(T)

– for each pixel of said at least a subset, identifying (120) such pixel as a candidate keypoint if, in such pixel, the approximation function has a local extreme (110) which is also a global extreme (116) with respect to the filtering parameter (σ) for a value σm of the filtering parameter in a respective sub-range of values of the filtering parameter internal to said predefined range said respective sub-range having a lower end larger than said lowest value of the filtering parameter and an upper end lower than said highest value of the filtering parameter, wherein the difference between said lowest value and said lower end, and the difference between said highest value and said upper end, are such that only the maximum or minimum points that happen in σm of which the behavior of the approximation function is known in a neighborhood of σm that is sufficiently large are considered as candidate keypoints;

– for each pixel identified as a candidate keypoint:

c) comparing (130) the value assumed by the approximation function at the value of the filtering parameter corresponding to the global extreme of the pixel with the values assumed by the approximation functions of the pixels adjacent to said pixel in the image at the values of the filtering parameters of the respective global extremes of such adjacent pixels, and

d) selecting (136) such pixel based on this comparison.

Is it patentable?

The Examining Division refused the application and considered the subject-matter of the claims lacked inventive steps starting from D1 (J Ng et al: “Steering in Scale Space to Optimally Detect Image Structures”) in view of D4 (D.G. Lowe: “Distinctive Image Features from Scale-Invariant Keypoints”).

The Board summarised the cited documents D1 and D4 as follows:

12. Document D1 describes a method of scale space approximation which is the same as that of the application (see D1, equations 5 to 15). This was also stated in the decision at point 2 and was not denied by the appellant.

12.1 The purpose of D1 is (introduction, page 483, paragraph 2) to address the “main problem … that exhaustive filtering with kernels over a wide range of finely-sampled scales is computationally intensive and inefficient”. The proposed method, by the formulation of what are called polynomial steering functions “lends itself well to the detection of global maxima by analytically finding the roots of the derivatives of the polynomials, rather than exhaustively performing operations over multiple scales to detect maxima as in the previous aforementioned works” (section 4, paragraph 1).

12.2 D1 uses global maxima over scales (section 4) to detect the scale of image structures in order to improve low-level feature detection and extraction by spatial filters (introduction). It is said though that the other polynomial roots “provide scale information about the other local energy maxima, minima and inflection points” (conclusion, last paragraph).

13. Document D4 represents the seminal work of Lowe regarding keypoint detection with scale invariant features. The method starts (section 3) by detecting local extrema in (DoG) scale space. These local extrema are obtained by comparing the value of a point with its neighbours both in the scale and the spatial direction (adjacent points – see figure 2). This raises the question of the sampling frequency in both scale and space domains (section 3.1). D4 uses experimentation in order to determine the frequency of sampling in the scale domain and “must settle for a solution that trades off efficiency with completeness” (end of section 3.1). To reduce computation, D4 further proposes to work on octaves (where the scale doubles), i.e. to repeat the method with the same set of scales, but on reduced versions of the image for each octave (see figure 1).

The applicant argued that document D1 was not a suitable starting document, but rather it was document D4.

16. The Board remarks that document D4 is commonly known in the field of image analysis. A person skilled in this art, i.e. the intended addressee of D1, would see that D1, although it mentions edges and ridges, proposes a general method of scale space approximation, which is said to be more computationally efficient than computation by discretization of the scale space. Furthermore, D1 teaches that with this approximation, for each image point, local and global extrema in scale can be computed analytically in an efficient way. So the skilled person would consider employing the method of D1 to improve the commonly known keypoint detection method of D4, which uses the standard scale space discretization.

17. That said, the Board accepts that this way of presenting the argument does not follow the established (but not mandatory) problem-solution approach, according to which the starting point would be the prior art to be improved and the objective technical problem to achieve that improvement. One does not, however, come to a different conclusion that way.

Therefore, the Board noted that irrespective of which document was used as the closest prior art, all features, except the last point of feature b, were obvious. However, the Board agreed with the applicant that the feature indeed provided a technical effect and acknowledged in the application as filed.

19. Thus the Board agrees in essence with the reasoning of the Examining Division regarding the combination of D1 and D4, though indeed the natural starting point according to the problem-solution approach is D4 rather than D1, because D4 is improved by D1 and not vice versa. It follows that all the claimed features but the last in feature b are obvious in view of said combination.

20. Regarding this last feature, the Examining Division and the appellant agree that this step solves a technical problem in that it allows to eliminate spurious or unstable candidate keypoints, due to the effects of the approximation.

20.1 The Board also notes that the Examining Division did not find this feature obvious over D1 and D4 but did not accept that the alleged technical effect could be acknowledged based on the application documents as filed.

20.2 The Board accepts this as a legitimate concern but finds to the contrary. The skilled person understands from page 17 that by keeping only the points for which “the behavior of the approximation functions are known in a neighborhood … that is sufficiently large”, the points outside the validity range of the approximation are eliminated, and that these are potentially not true extrema, and hence no real keypoints, but are due to errors in the approximation. Thus the technical problem of eliminating “false” extrema is clear from the application.

Therefore, the Board concluded that the subject-matter of the claim involves an inventive step, and the appeal is allowed with an order to grant a patent

More information

You can read the whole decision here: T 1482/19 (Scale space approximation/TELECOM ITALIA) of February 11, 2022, of the Technical Board of Appeal 3.5.06

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

www.bardehle.com/de/team/richard-preston

Pedestrian simulation: non-technical

This decision relates to a European patent application that concerns simulating pedestrian crowd movement, which was a subject of the referral to the Enlarged Board of Appeals in G 1/19. The Board decided that the behaviour of a crowd moving through an environment does not contribute to a technical effect as the potential use of such data is not limited to technical purposes, as it can be used in computer games or presented to a human for obtaining knowledge about the modelled environment. Similarly, the method of designing a building structure was also not technical as it was entirely up to the human designer as to how to revise the model and the step of measuring attributes of pedestrians cannot independently support an inventive step.

Here are the practical takeaways from the decision T 0489/14 (Pedestrian simulation/BENTLEY SYSTEMS) of November 26, 2021 of the Technical Board of Appeal 3.5.07:

Key takeaways

In the case of a computer-implemented invention, a technical effect relevant for the assessment of inventive step exists if the features of the claim directly achieve a (real) technical effect on physical reality (including both external physical reality and the “internal” physical reality of the computer system in which the invention is implemented)

The behaviour of a crowd moving through an environment does not contribute to a technical effect as the potential use of such data is not limited to technical purposes, as it can be used in computer games or presented to a human for obtaining knowledge about the modelled environment.

A design process was normally a cognitive activity. Since the data representing a model of a building structure may be used for a variety of non-technical purposes, for example in a video game, and therefore has no “implied technical use”.

Since the claim leaves it entirely up to the human designer as to how to revise the model, the simulation does not influence the designed model in any technical manner and thus does not contribute to a technical effect.

The technical step of measuring attributes of pedestrians cannot independently support an inventive step and does not interact with the other features of the claim to provide a combined technical effect.

The invention

The patent application underlying this case concerns the modelling of pedestrian movement, which could be used to help design or modify a venue (building structures like houses or train stations). According to the applicant, this provides a more accurate and realistic simulation of pedestrian crowds in real-world situations, which could not be adequately modelled by conventional simulators. The invention is said to be based on the insight that human interaction could be expressed and modelled in the same way as physical interactions.

Claim 1 of the Main Request

A computer-implemented method of modelling pedestrian crowd movement in an environment, the method comprising:

simulating movement of a plurality of pedestrians through the environment, wherein simulating movement of each pedestrian comprises:

providing a provisional path (9) through a model of the environment from a current location (6) to an intended destination (7);

providing a profile for said pedestrian;

determining a preferred step (112′), to a preferred position (123′), towards said intended destination based upon said profile and said provisional path, wherein determining said preferred step comprises determining a dissatisfaction function expressing a cost of taking a step comprising a sum of an inconvenience function expressing a cost of deviating from a given direction and a frustration function expressing a cost of deviating from a given speed;

defining a neighbourhood (29) around said preferred position (123′);

identifying obstructions in said neighbourhood, said obstructions including other pedestrians (21) and fixed obstacles (25);

determining a personal space (24) around said pedestrian;

determining whether said preferred step (112′) is feasible by considering whether obstructions (21, 25) infringe said personal space over the course of the preferred step (112′)

The Referral (G 1/19)

The present appeal was subject to an interlocutory decision on 22 February 2019, where the Board had raised doubt whether a method of modelling pedestrian crowd movement in an environment and using the results for designing a building structure is technical or not. According to earlier case law referring to the simulation of an electronic circuit, a computer-implemented method has technical character if the method is functionally limited to a technical purpose. However, the Technical Board of Appeal was uncertain whether the earlier reasoning is correct since the claimed subject-matter allegedly lacks a link to the physical reality.

Hence, the Board formulated three referral questions to the Enlarged Board of Appeal (the highest judicial entity in the European patent system) asking it to decide essentially whether the simulation of a technical system has to be excluded from patent protection or not.

This was discussed in our previous article Simulating pedestrian crowd movement: to be decided by the Enlarged Board of Appeal, with the practical takeaways from the interlocutory decision of T 0489/14 (Pedestrian simulation/CONNOR) of 22.2.2019

The Enlarged Board of Appeals then handed down its decision in G 1/19 on the patentability of simulation of Computer-implemented simulation. This decision was also discussed in detail in the video IP Expert Talk: Decision G 1/19, see below:

The referring Board summarised the answers to the referral G 1/19 as follows:

2.3 The order of decision G 1/19 answers the first question in the affirmative, i.e. a computer-implemented simulation of a technical system or process that is claimed as such, i.e. on its own, can, for the purpose of assessing inventive step, solve a technical problem by producing a technical effect going beyond the simulation’s implementation on a computer.

With respect to the second question, the order only states that for that assessment it is not a sufficient condition that the simulation is based, in whole or in part, on technical principles underlying the simulated system or process.

The Enlarged Board refrained from giving an exhaustive list of criteria for assessing whether a computer-implemented simulation solves a technical problem by producing a technical effect that goes beyond the simulation’s implementation on a computer, but it did give further guidance.

2.4 In point 88 of its decision, the Enlarged Board considered that a direct link with external physical reality was not always necessary. On the one hand, technical contributions could be established by features within the computer system. On the other hand, there were many examples in which potential technical effects, which could be distinguished from direct technical effects on physical reality, had been considered in the course of the technicality/inventive-step analysis. Moreover, the notion of technicality had to remain open.

2.5 With respect to potential technical effects, the Enlarged Board added that if claimed data or data resulting from a claimed process was specifically adapted for the purpose of controlling a technical device, the technical effect that would result from this intended use of the data could be considered “implied” by the claim; however, this argument could be made only if the data had no other relevant uses, since otherwise the technical effect was not achieved over substantially the whole scope of the claimed invention (G 1/19, points 94 and 95).

Is it patentable?

Based on the guidance from the referral G 1/19, the Board concluded that the computer-implemented method of modelling pedestrian crowd movement in an environment (Main Request) did not contribute to the technical effect :

2.6 The Enlarged Board also dealt with the argument that “virtual or calculated technical effects” which were not achieved through an interaction with physical reality, but were calculated so as to correspond closely to “real” technical effects of physical entities, should be treated as technical effects for the purpose of assessing inventive step (using the COMVIK approach).

The Enlarged Board distinguished such “virtual or calculated” effects from potential technical effects which necessarily became real technical effects, for example when a computer program or a control signal for an image display device was put to its intended use (G 1/19, point 97).

It explained that calculated information reflecting the status or physical properties of a physical object was, first and foremost, mere data which could be used in many ways. In exceptional cases it could have an “implied technical use” which could be the basis for an “implied technical effect“. A claim with no (implied) limitation to a technical use would routinely raise concerns with respect to the principle that the claimed subject-matter had to be a technical invention over substantially the whole scope of the claim (G 1/19, point 98; see also points 124 and 128).

2.7 Hence, in the case of a computer-implemented invention, a technical effect relevant for the assessment of inventive step exists if the features of the claim directly achieve a (real) technical effect on physical reality (including both external physical reality and the “internal” physical reality of the computer system in which the invention is implemented).

In addition, an “implied” technical effect relevant for the assessment of inventive step is present if the claimed invention or the data produced by it necessarily achieves a real technical effect when it is put to its intended (and only relevant) use.

In contrast, merely providing calculated data which corresponds closely to technical effects of physical entities is not a technical effect relevant for the assessment of inventive step.

2.8 It follows that the data produced by the method of claim 1, which reflects the behaviour of a crowd moving through an environment, does not contribute to a technical effect for the purpose of assessing inventive step. Indeed, the potential use of such data is not limited to technical purposes, as it can be used in computer games or presented to a human for obtaining knowledge about the modelled environment, to give just two examples of non-technical uses that are within the scope of the claim.

2.9 The subject-matter of claim 1 of the main request therefore lacks inventive step (Article 56 EPC).

The Board then considered the fourth auxiliary Request which relates to the method of designing a building using the pedestrian simulation.

Claim 1 of the Fourth Auxiliary Request - amendments emphasised in Bold

A method of designing a building structure, the method comprising:

providing a model of said building structure;

simulating movement of a plurality of pedestrians through said building structure using a computer, wherein simulating movement of each pedestrian step comprises:

providing a provisional path (9) through a model of the environment from a current location (6) to an intended destination (7);

providing a profile for said pedestrian;

determining a preferred step (112′), to a preferred position (123′), towards said intended destination based upon said profile and said provisional path, wherein determining said preferred step comprises determining a dissatisfaction function expressing a cost of taking a step comprising a sum of an inconvenience function expressing a cost of deviating from a given direction and a frustration function expressing a cost of deviating from a given speed;

defining a neighbourhood (29) around said preferred position (123′);

identifying obstructions in said neighbourhood, said obstructions including other pedestrians (21) and fixed obstacles (25);

determining a personal space (24) around said pedestrian;

determining whether said preferred step (112′) is feasible by considering whether obstructions (21, 25) infringe said personal space over the course of the preferred step (112′); and

revising said model of said building structure in dependence upon movement of the pedestrians.

4.1 In claim 1 of the fourth auxiliary request, the simulation method of claim 1 of the third auxiliary request is claimed as part of a design process. The claim includes a step of revising a designed model in dependence upon the simulated movement of pedestrians through the modelled building structure. The description of the application makes it clear that the revising step may be performed by a human designer operating a CAD program (see e.g. page 70, lines 7 to 9, of the published application).

4.2 According to the order of decision G 1/19, the answers to the first and second questions are no different if the computer-implemented simulation is claimed as part of a design process, in particular for verifying a design.

The Enlarged Board considered that a design process was normally a cognitive activity (G 1/19, point 143). It saw no need for the application of special rules if a simulation was claimed as part of a design process (point 144).

4.3 Since the added revising step may be carried out by a human, an inventive step in the subject-matter of claim 1 still cannot reside in the implementation of the method on a computer.

4.4 In addition to producing data reflecting the behaviour of a crowd moving through an environment, now as an intermediate result, the method of claim 1 produces data representing a (revised) model of a building structure.

4.5 The board notes that data representing a model of a building structure may be used for a variety of non-technical purposes, for example in a video game, and therefore has no “implied technical use“.

For this reason alone, the claimed simulation still does not contribute to a technical effect.

4.6 The board also notes that the simulation does not influence the designed model in any technical manner, since the claim leaves it entirely up to the human designer as to how to revise the model “in dependence upon movement of the pedestrians”. This is another reason why the simulation in the context of claim 1 of the fourth auxiliary request does not contribute to a technical effect.

4.7 Hence, the subject-matter of claim 1 of the fourth auxiliary request lacks inventive step (Article 56 EPC).

The applicant also filed claims as a seventh auxiliary Request which ensured that the invention used measured data as input data.

Claim 1 of the Seventh Auxiliary Request - amendments emphasised in Bold

A computer-implemented method of modelling pedestrian crowd movement in a building structure, the method comprising:

simulating movement of a plurality of pedestrians through the building structure, wherein simulating movement of each pedestrian comprises:

providing a provisional path (9) through a model of the building structure from a current location (6) to an intended destination (7);

providing a profile for said pedestrian;

determining a preferred step (112′), to a preferred position (123′), towards said intended destination based upon said profile and said provisional path, wherein determining said preferred step comprises determining a dissatisfaction function expressing a cost of taking a step comprising a sum of an inconvenience function expressing a cost of deviating from a given direction and a frustration function expressing a cost of deviating from a given speed;

defining a neighbourhood (29) around said preferred position (123′);

identifying obstructions in said neighbourhood, said obstructions including other pedestrians (21) and fixed obstacles (25);

determining a personal space (24) around said pedestrian;

determining whether said preferred step (112′) is feasible by considering whether obstructions (21, 25) infringe said personal space over the course of the preferred step (112′)

wherein providing said profile for said pedestrian comprises:

basing said profile on a set of measured attributes.

7.1 The appellant submitted that the features added to claim 1 of the seventh and eighth auxiliary requests ensured that the claimed invention used measured data as input data. The Enlarged Board had pointed out that technical effects could occur at the input of a computer-implemented process and that technical input could consist of a measurement. Moreover, it had been recognised in decision T 1892/17, Reasons 2.1.4, that a simulation based on measurements did not produce a purely virtual effect. The added features had the effect of providing a more accurate simulation.

7.2 The board first notes that, even if the added features credibly improved the accuracy of the simulation, this alone could not establish a technical contribution made by the simulation. As the Enlarged Board stated in point 111 of its decision, whether a simulation contributes to the technical character of the claimed subject-matter does not depend on the quality of the underlying model or the degree to which the simulation represents “reality”.

7.3 Claim 1 of the seventh auxiliary request does not include an actual step of measuring but only specifies that the attributes of the pedestrian profile have been measured, i.e. obtained by some measurement in the past. It is questionable whether this feature has a limiting effect, since whether data has been generated by a measurement or by some other method is not a property of the data itself.

However, for the sake of argument, the board will interpret claim 1 of the seventh auxiliary request as including a step of generating the attributes of the pedestrian profile by means of measurements, and it will assume that these measurements are carried out using technical means (and not just by observing and counting pedestrians passing through various building structures).

7.4 With this interpretation, the method of claim 1 consists of a technical step of measuring attributes of pedestrians to provide pedestrian profiles followed by the (computer-implemented but otherwise) non-technical simulation method of claim 1 of the first auxiliary request.

The question to be answered is whether the technical step and the simulation method are not merely juxtaposed, the output of the measuring step serving as an input for the simulation method, but interact to produce a combined technical effect (see decision T 154/04, OJ EPO 2008, 46, Reasons 5, under (F)). An interaction may be present, for example, if the combination amounts to an indirect measurement of a specific physical entity by means of measurements of another physical entity (see G 1/19, point 99). This is in line with decision T 1892/17, which found that the simulation features interacted with the technical features of the claim to contribute to a specific technical effect (see Reasons 2.1.4).

7.5 The method of claim 1 provides information about the movement of simulated pedestrians through a modelled building structure. Since the calculated information is neither used in a further step of the method nor specifically adapted for the purposes of an intended technical (and only relevant) use, it has to be investigated whether the information represents a measurement of a physical entity.

The modelled building structure does not correspond to a building structure that was measured, whether directly or through measurement of its physical effects on other physical entities. In fact, the modelled building structure need not correspond to any existing building structure.

Nor does the calculated information about the movement of simulated pedestrians represent a direct or indirect measurement of any of the real pedestrians (or other physical entities) that were measured in the process of generating the pedestrian profiles. The group of simulated pedestrians does not necessarily correspond to the set of measured entities, as their profiles need only be “based” on a set of measured attributes. For example, the value for the “preferred walking speed” attribute in each pedestrian profile may have been randomly drawn from a probability distribution derived from a statistical analysis of actual measurements of a large number of individuals (as described on page 15, lines 10 to 19, of the description). In this case, none of the modelled pedestrians corresponds to any of the pedestrians involved in the measurements.

Hence, in the present case no physical entity (or process) can be identified which could potentially be measured by the method of claim 1 in the sense that its physical status or some physical property is described by information calculated on the basis of data obtained by a direct or indirect physical interaction with the entity.

It therefore does not need to be determined whether the information calculated by the method of claim 1 – essentially the trajectories of simulated pedestrians moving through a modelled environment – is of the kind that could describe such a technical status or properties.

7.6 Since the technical step of measuring attributes of pedestrians cannot independently support an inventive step and does not interact with the other features of the claim to provide a combined technical effect, the subject-matter of claim 1 of the seventh auxiliary request lacks inventive step (Article 56 EPC).

The applicant had also filed claims with a tenth and eleventh Auxiliary request which relates to a method of controlling the movement of an “autonomous entity” through an environment, the entity being a robot (such as a cleaning robot for example).

However, these claims were different from those discussed and were originally dropped when entering EP regional phase. Therefore, the Board did not admit the requests as they considered it to represent a switch of the claimed invention which was deliberately not prosecuted in the first-instance proceedings.

Therefore, the appeal was dismissed and the patent application was refused.

More information

You can read the whole decision here: T 0489/14 (Pedestrian simulation/BENTLEY SYSTEMS) of November 26, 2021 of the Technical Board of Appeal 3.5.07

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

www.bardehle.com/de/team/richard-preston

Selective haptic feedback: technical

This decision relates to a European patent application that concerns an electronic device with a touch-sensitive display surface to detect gestures and provide tactile output and forgoing tactile feedback when making a zoom gesture. The Board agreed with the applicant that forgoing tactile feedback and providing selective haptic feedback improves the operations and feedback provided during interaction with the device is a technical problem. Since tactile feedback would potentially interfere with the user’s ability to make an ongoing zoom input gesture and be disturbing, forgoing tactile feedback when making a zoom gesture is a plausible advantage. Here are the practical takeaways from the decision T 1645/19 (Selective haptic feedback/APPLE) of January 26, 2022 of the Technical Board of Appeal 3.5.05:

Key takeaways

Improving the operations and feedback provided during interaction with the device is a technical problem.

Forgoing tactile feedback when making a zoom gesture is advantageous, since tactile feedback would potentially interfere with the user’s ability to make an ongoing zoom input gesture and be disturbing.

Cherry-picking different aspects of cited document for a combination that is not disclosed and for which there is no pointer would be based on hindsight knowledge of the invention.

The invention

The invention relates to an electronic device with a touch-sensitive display surface to detect gestures and provide tactile output.

Many electronic devices provide a form of confirmation to a user in response to an event being triggered by a user action. However, this confirmation or feedback can be distracting or confusing to a user when it occurs in response to inputs that do not correspond to the feedback.

For instance, the user may be confused between tactile feedback provided with the single contact (such as for selection) and a multitouch contact (such as for pinch zoom input).

The invention provides a more convenient and intuitive interface by generating a tactile output in response to detecting a gesture that includes a first number of contacts (e.g., one contact) and forgoing generating the tactile output if the gesture includes a second number of contacts (e.g., two or more contacts).

Fig. 17F of WO 2014/105275 A1

Claim 1 of the Main Request

A method (10000), comprising:

at an electronic device (300) with a display (450) and a touch-sensitive surface (451),

wherein the device includes one or more sensors (357) to detect intensity of contacts with the touch-sensitive surface:

detecting, on the touch-sensitive surface, a gesture that includes an increase of intensity of a contact above a respective intensity threshold (10002); and

in response to detecting the gesture (10008), triggering an event in accordance with the gesture:

in accordance with a determination that the gesture includes a first number of contacts, generating a tactile output on the touch-sensitive surface (10010); and

in accordance with a determination that the gesture includes a second number of contacts different from the first number, forgoing generating the tactile output on the touch-sensitive surface (10018) and performing an operation of reducing a size of an object displayed on the display in accordance with the gesture including the second number of contacts,

wherein the first number of contacts is one contact (10012), and the second number of contacts is two contacts (10020).”

Is it patentable?

The first-instance Examining Division decided that the subject-matter of claim 1 lacked inventive step starting from the disclosure of D1: EP 2 375 314. The Board identified the distinguishing features as follows:

2.2 D1 does not disclose the following features of claim 1:

– the device performs an operation in response to detecting a gesture including two contacts and including an increase of intensity of a contact above the threshold for triggering an event,

– this operation consists in reducing the size of an object displayed in accordance with the gesture, and

– for this two-contact gesture, generation of tactile output is forgone.

The technical effect of these distinguishing features is that an operation of zoom-out is enabled by a two-contact gesture, without any tactile feedback.

The objective technical problem can thus be formulated as how to improve the operations and feedback provided during interaction with the device.

In the preliminary opinion, the Board indicated that the features were not inventive as the cited document D1 provided a threshold for providing feedback. Therefore, the combination of detection and forgoing feedback were obvious. Furthermore, the advantages were said to be heavily dependent on the skills of the user.

However, during the Oral Proceedings, the applicant overcame these objections by arguing that cherry-picking different embodiments were in hindsight of the invention and the Board agreed that the advantages were plausible:

2.3 At the priority date of the present application in 2012, the skilled person was aware that touch-screen devices able to detect multi-touch gestures, i.e. multi-finger contacts, such as the device of D1, could be adapted to detect the so-called “pinch-to-zoom” gesture. It was thus common ground in the oral proceedings that applying to the device of D1 recognition of the pinch-to-zoom gesture and its associated functionality did not contribute to an inventive step.

However, only in one passage does D1 disclose forgoing tactile feedback when an operation is performed after a touch (see paragraph [0057]. In that case, a lower touch threshold and an upper touch threshold are defined for a location, and a touch having an intensity between the lower and the upper touch thresholds results in the operation of highlighting a selection option, but without tactile feedback. The teaching of paragraph [0057] therefore implies that for a one-contact gesture, the performing of an operation may occur with tactile feedback. However, claim 1 specifies that for all the locations on the touch screen the same intensity threshold, i.e. force threshold, triggers an event both in the case of a one-contact and in the case of a two-contact gesture, whereas tactile feedback is always generated in the case of a one-contact gesture. Thus, nothing in the teaching of D1 in paragraph [0057] provides any hint to the skilled person of implementing the forgoing of tactile feedback only in the case of a two-contact gesture as defined in claim 1.

Furthermore, D1 discloses in paragraphs [0033] and [0043] the forgoing of tactile feedback in case of a multi-contact gesture, but only on the assumption that this gesture was made inadvertently and is therefore not to trigger any operation. Thus, the teaching of D1 in paragraphs [0033] and [0043] does not provide any hint to the skilled person of implementing the forgoing of tactile feedback in the case of a two-contact gesture triggering an operation, as defined in claim 1.

Moreover, even if paragraph [0042] of D1 were interpreted as meaning that a touch force threshold may depend on the number of simultaneous touches, i.e. that a force threshold for a two-contact gesture could be made different from a force threshold for a one-contact gesture, this passage of D1 would not provide the skilled person with any hint with respect to forgoing tactile feedback depending on the number of touches.

Furthermore, the board agrees with the appellant that cherry-picking different aspects of D1 for a combination that is not disclosed and for which there is no pointer in D1 – namely implementing a touch threshold for generating tactile feedback – and then choosing a number of contacts as the basis for the threshold even when a gesture is positively identified and a subsequent action taken would be based on hindsight knowledge of the invention.

The appellant also plausibly argued that forgoing tactile feedback when making a zoom gesture, as defined in claim 1, is advantageous, since the user might exceed the contact intensity threshold at any point during the zoom operation. In that case, tactile feedback would potentially interfere with the user’s ability to make an ongoing zoom input gesture and be disturbing. Moreover, forgoing tactile feedback in such circumstances will also help to save battery charge for portable electronic devices.

Therefore, the subject-matter of the claims was considered to involve an inventive step, and the appeal is allowed with an order to grant a patent.

More information

You can read the whole decision here: T 1645/19 (Selective haptic feedback/APPLE) of January 26, 2022 of Technical Board of Appeal 3.5.05

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

www.bardehle.com/de/team/richard-preston

Dynamically changing geo-fence for a device: technical

This decision relates to a European patent application that concerns providing geo-fences to provide a notification to the mobile device. The Board considered that dynamically changing the effective geo-fence boundary solely for that mobile device based on the number of applications in the device, reduces the load for the mobile device effectively and prevents overload. Since this characteristic of the mobile device is a permanent characteristic of the device even if this characteristic becomes evident only in certain overload circumstances, it provides a credible technical effect. Here are the practical takeaways from the decision T 2520/19 () of February 02, 2022 of Technical Board of Appeal 3.4.02:

Key takeaways

Dynamically changing the effective geo-fence boundary solely for that mobile device is technical

The claimed mobile device with its particular treatment of geo-fences in case it is executing a large number of applications that actively utilise geo-fences reduces the load for the mobile device effectively and prevents overload.

The technical effect was credibly achieved since this characteristic of the mobile device is a permanent characteristic of the device even if this characteristic becomes evident only in certain overload circumstances.

The invention

The invention relates to geo-fences to provide a notification to the mobile device. In order to focus the notification on the most promising customers, an entity, such as a restaurant, may establish a geo-fence that determines which users may receive the notification. For example, only users that enter the defined area encompassed by the geo-fence boundary may receive the notification.

The geo-fence may be defined in a variety of configurations, such as a circular area defined by a radius around the location of the restaurant, and may be based, for example, on longitude and latitude coordinates. In a locality with a high density of geo-fences. It may cause an overloaded of notifications.

In order to prevent a mobile device from being overloaded with notifications, the geo-fence is processed differently when a mobile device is executing a large number of applications (that are actively utilizing geo-fences) by encompass a smaller area than a geo-fence when a mobile device or a plurality of mobile devices comprise a small number of applications.

Fig. 1 of WO 2015/085176 A1

Claim 1 (Mobile Device) of the Sole Request

1. A mobile device (104-1 – 104-N) for use with an apparatus (108-1 – 108-N, 200) for generating a plurality of geo-fences to provide a notification to the mobile device (104-1 – 104-N) when the mobile device (104-1 – 104-N) enters or resides in an area encompassed by a geo-fence, the apparatus (108-1 – 108-N, 200) being configured to obtain a trigger generated in response to a change in a geo-fence criteria, and to define one or more boundaries of each geo-fence based on the geo-fence criteria in response to obtaining the trigger;

the mobile device (104-1 – 104-N) being configured to:

process the plurality of geo-fences differently based on a number of applications that actively utilize geo-fences on the mobile device (104-1 – 104-N) to prevent the mobile device (104-1 – 104-N) from being overloaded with notifications for geo-fences, wherein, when the mobile device (104-1 – 104-N) is executing a large number of applications that are actively utilizing geo-fences and crosses a boundary of a geo-fence, the mobile device (104-1 – 104-N) is configured to delay recognition of the geo-fence until the mobile device ( 104-1 – 104-N) is closer to a center of an area defined by the geo-fence so that the recognized geo-fence encompasses a smaller area than the geo-fence when the mobile device ( 104-1 – 104-N) is executing a small number of applications that actively utilize geo-fences.

Is it patentable?

The first-instance Examining Division decided that the subject-matter of claim 1 lacked inventive step starting from the disclosure of D4: US 2012/172027 A1. The distinguishing features were identified as follows:

4.4 The board agrees with the examining division and the appellant that the claimed invention differs from the disclosure of document D4 in that the mobile device is configured to process the plurality of geo-fences differently based on a number of applications that actively utilize geo-fences on the mobile device to prevent the mobile device from being overloaded with notifications for geo-fences, wherein, when the mobile device is executing a large number of applications that are actively utilizing geo-fences and crosses a boundary of a geo-fence, the mobile device is configured to delay recognition of the geo-fence until the mobile device is closer to a center of an area defined by the geo-fence so that the recognized geo-fence encompasses a smaller area than the geo-fence when the mobile device is executing a small number of applications that actively utilize geo-fences.

4.5 These differing features provide the effect of preventing the mobile device from being overloaded with notifications (see published application, paragraph 0031).

4.6 The board agrees with the examining division that the objective technical problem to be solved by the invention is therefore preventing the mobile device from being overloaded with notifications.

In the first instance, although the Examining Division agreed with the distinguishing features and the technical problem, they disagreed with the applicant that the claimed effect was indeed achieved.

4.7 The examining division was of the opinion that the claim did not define any kind of geographical relationship between the geo-fences that were monitored by the mobile device, which implied that said geo-fences were, in most of the cases, distributed in such a way that the mobile device did not receive a plurality of notifications when it crossed a boundary of one of the geo-fences. As a result, the technical effect of preventing the mobile device from being overloaded with notifications was not achieved in the majority of the cases that were covered by the claim. The solution proposed in claim 1 of the present application therefore could not be considered to involve an inventive step (see contested decision, section 3.2, last two paragraphs).

The applicant successfully argued against this decision, and the Board agreed with the applicant.

4.8 The appellant argued that the local processing effect on the mobile device brought about the technical effect of dynamically changing the effective geo-fence boundary solely for that mobile device. In this way, a single geo-fence generated by a geo-fence system centrally could have a diverse effect on the geo-fence processing by a plurality of mobile devices. When a mobile device was already actively utilizing a plurality of geo-fences, overloading of the processing of notifications by the mobile device could be ameliorated by delaying recognition of a geo-fence to effectively reduce its area. The effect was provided also for non-overlapping areas of geo-fences when the mobile device was moving fast. By delaying recognition of a geo-fence, an overload of processing of notifications could be avoided by providing a greater time period between notifications. For overlapping, partially overlapping and separate geo-fences, the technical problem of notification overload could be experienced by the mobile device when moving relative to the geo-fences. There was thus a credible technical effect for a number of configurations of geo-fences and mobile devices that fell within the scope of the claimed invention.

4.9 The board shares the opinion of the appellant. The claimed mobile device with its particular treatment of geo-fences in case it is executing a large number of applications that actively utilise geo-fences reduces the load for the mobile device effectively and prevents overload. This characteristic of the mobile device is a permanent characteristic of the device even if this characteristic becomes evident only in certain overload circumstances. The board is therefore of the opinion that the claimed subject-matter represents a technical invention that involves technical means to solve the technical problem. The claimed subject-matter is not suggested by document D4 or any other prior art document cited by the examining division.

Therefore, the subject-matter of the claims was considered to involve an inventive step, and the appeal is allowed with an order to grant a patent.

More information

You can read the whole decision here: T 2520/19 () of February 02, 2022, of Technical Board of Appeal 3.4.02

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

www.bardehle.com/de/team/richard-preston

Dynamically offering a user interface option on meeting certain conditions: technical

This decision relates to an Opposition appeal against a granted European patent that concerns providing an option on a user interface with an aim to simplify and speed up the use of a device. The Board considered that dynamically offering a user interface option in response to an image been captured during a telecommunication connection with another device credibly assists the user in quickly sending the captured image information to the other device. This avoids the need to select the captured image information from the memory and to specify data identifying the other device. Here are the practical takeaways from the decision T 0282/17 of December 15, 2021 of Technical Board of Appeal 3.5.04:

Key takeaways

Offering the user of a telecommunication device an option in response to image information having been captured during a telecommunication connection with another device credibly assists the user in quickly sending the captured image information to the other device by avoiding the need to select the captured image information from the memory and to specify data identifying the other.

Dynamically offering a user selectable option in response to a determination that certain conditions are met is technical.

Improving the sending of image information from one device to another is a technical problem.

The invention

The invention relates to a method and a system to simplify and speed up the use of a telecommunication device in connection with sending various kinds of multimedia messages from one device to another advantageously when the device is in communication connection with the other device.

For example, when a user takes an image with a camera of the device, the device identifies the mode of the device at that moment (is a call active or not) and provides a user interface with having the possibility to send the taken image straight to the device with which the communication connection had been made (for example, from A-subscriber to B-subscriber).

Fig. 1a of EP 1 755 328 A1

Claim 1

A method comprising: establishing a wireless telecommunication connection between a first telecommunication device and another telecommunication device,

– creating, using a camera, image information in the first telecommunication device during the telecommunication connection to the another telecommunication device;

– offering on a user interface of the first telecommunication device, in response to a determination that image information has been created using the camera during said telecommunication connection, a user selectable option to send the created image information from the first telecommunication device to the another device with which the telecommunication connection has been established; and

sending, in response to user selection of the user selectable option, the created image information from the first telecommunication device to the another device, while the telecommunication connection between the first telecommunication device and the another telecommunication device is still active.

Is it patentable?

The first-instance Opposition Division found that the subject-matter of claim 1 was inventive. The Opponent appealed the decision and alleged, in addition to other objections, that the subject-matter lacked novelty over the disclosure of E1 (EP 1 148 725 A2), or at least, lacked an inventive step starting with document E1 and common general knowledge.

According to the Opponent-Appellant, document E1 is related to a videophone apparatus taking a live picture of the user and provides the two keys which when pressed capture a live picture for the videophone. This implicitly disclosed the case where image information was sent after one of the keys was pressed.

The Board did not agree with this argument of the Opponent-Appellant, and in agreeing with the Patentee-Respondent agreed that the subject-matter was novel for the following reasons:

6.2 The board agrees with the respondent that, when interpreted properly in the context of the patent, claim 1 specifies dynamically offering a user selectable option in response to a determination that certain conditions are met (see point XIX.(a) above). Interpreting claim 1 as encompassing the case where the user selectable option was offered permanently (see appellant’s argument under point XIX.(e) above) would mean that the expression “in response to a determination that image information has been created using the camera during said telecommunication connection” would have no limiting effect on its subject-matter. This would be inconsistent with the embodiments of the description, which all teach automatically triggering the offering of a user selectable option once it is determined that the user has captured image information and that the device is in a state of data connection with another device (see the description of steps 101 to 103 of Figure 1a and steps 111 to 113 of Figure 1b in the passage from page 2, line 24 to page 4, line 6 of the application as filed and paragraphs [0015] and [0016] of the patent, as well as the paragraph bridging pages 6 and 7 of the application as filed and paragraph [0021] of the patent).

6.3 The board agrees with the opposition division (see point 15.1 of the decision) and the respondent (see point XIX.(b) above) that the offering of the two keys 111b and 111c described in document E1 is not dynamically triggered by the creation of image information during the telecommunication connection. These keys are always available on the user interface and can be pressed by the user at any time to trigger the sending of either a live picture (key 111b) or a processed picture (key 111c). The last sentence of paragraph [0101] of E1, referred to by the appellant (see point XIX.(f) above), merely describes the possibility that the user can start sending their live picture when they are ready to show themselves to their interlocutor.

The Opponent-Appellant then argued that the distinguishing feature of dynamically offering a user interface is not inventive as it merely relates to providing a soft button graphical user interface instead of a physical key.

The Board disagreed, and in agreement with the Patentee-Respondent acknowledged that the feature is technical:

7.3 The board disagrees with the appellant that the only technical effect achieved by the features identified under point 6.5 above is that the offering is represented not by a physical key but by a soft button on a graphical user interface (see point XX.(a) above). Offering the user of a telecommunication device an option in response to image information having been captured during a telecommunication connection with another device credibly assists the user in quickly sending the captured image information to the other device by avoiding the need to select the captured image information from the memory and to specify data identifying the other device.

7.4 The technical background of the patent is a telecommunication device establishing a voice connection with another device (see paragraph [0003]). As stated in paragraph [0010] of the patent, the purpose is to simplify and speed up the sending of various kinds of multimedia messages from one device to another when the device is in communication-connection with the other device. As indicated under point 7.3 above, this problem is solved by the subject-matter of claim 1.

7.5 In view of this, the board finds it justified to formulate the objective technical problem as improving the sending of image information from one device to another, as proposed by the respondent (see point XX.(d) above).

7.6 The board agrees with the respondent (see point XX.(g) above) that even if it were to be considered obvious to replace the physical keys described in E1 with software keys and to dynamically offer the software keys on the basis of the conditions specified in claim 1 (see the appellant’s arguments under point XX.(b) and (c) above), the person skilled in the art would have found no prompt in E1 to modify the functionality of the software key in such a way that selecting it triggered the sending of the retrospective image information whose creation generated (or activated) the software key, as required by claims 1 and 4 of the revised second auxiliary request.

7.7 This reason is sufficient to arrive at the conclusion that the subject-matter of claims 1 and 4 involves an inventive step in view of the disclosure of document E1 combined with the common general knowledge of the person skilled in the art.

Therefore, claim 1 was considered to involve an inventive step, and the appeal is allowed with an order to maintain the patent (in amended form).

More information

You can read the whole decision here: T 0282/17 of December 15, 2021, of Technical Board of Appeal 3.5.04

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

www.bardehle.com/de/team/richard-preston

Cinematographic image interface: technical

This decision relates to a European patent application that concerns the process for making an audio-visual interface. The Board disagreed with the first instance division and considered that the technical difference in shooting technology, and the implementation of the obtaining and projection method was neither a common measure nor mere presentation of information, but its use was not obvious from the prior art and thus exceeded the general design capabilities of skilled person. Here are the practical takeaways from the decision T 0799/19 (Cinematographic image interface/BLUE CINEMA) of October 11, 2021, of Technical Board of Appeal 3.5.05:

Key takeaways

Even if it were to be assumed that a feature (shooting a subject) in itself was a common measure, its use in the prior art may not be obvious for the skilled person due to the technical differences between the prior art

Even if it is considered that obtaining and projecting an image is an obvious measure and that the image provided is only a different presentation of a virtual agent to the user with no technical effect, implementing the image-obtaining and image-projecting method in the prior art would necessitate structural modifications of the device of the prior art which exceed the general design capabilities of the skilled person

The invention

The invention refers to a process for making an audio-visual interface. It aims to provide an interactive real human being (filmed and not generated with computer graphics), and later reproduced with a three-dimensional effect to increase the visual involvement.

For this, the invention shoots the subject with a shooting machine of specific technology, such that images of the subject are obtained with a specific requirement with the “edge cutting” effect, as in Fig, 5. These recorded images are then reproduced on a plane made of transparent polycarbonate placed at 45° as in Gif. 8, using specific method steps.

Fig. 5 and 8 of EP 2 965 172 A1

Claim 1 (main request)

Process for making an audio-visual interface which reproduces a complete interactive human being as a chosen subject (7), the process, comprising the steps of:

– interactivity analysis;

– image acquisition; and

– reproduction, through an optical or television system with voice, gesture, logic interaction capabilities,

wherein the step of interactivity analysis comprises the sub-steps of: using an optical apparatus which reproduces with three-dimensional effect the human being, equipped with a sensor/controller (3) capable of performing a voice recognition, able to interpret the human language, the interpretation of the human language allowing to identify the information semantics; triggering the appropriate event depending on the received voice command, communicating with a computer through an interface to connect external devices; perceiving the presence, absence and related status change, and a series of movements by a viewer/user (1), and

wherein the chosen subject (7) recites all provided segments, namely ACTION, and a series of non-verbal actions, namely IDLE, PRE-IDLE and BRIDGE, which provide the interface with a presence effect due to the simulation of the human behavior during the dialogue, the visual sequences to shoot being segmented according to an interactivity logic to obtain a sensation of talking with a human being, the logic sequences being divided into four main categories: 1) IDLE sequence (indifference status in which the interface “lives” waiting for a status change command); 2) PRE-IDLE sequence (a behavior which occurs before the action); 3) BRIDGE sequence (an image transition which helps giving visual continuity to the interface between the IDLE/PRE-IDLE status and the ACTION status); and 4)ACTION sequence (in which the interface performs the action or makes the action performed), wherein:

– the step of image acquisition provides for the use of a cinematographic technique comprising the sub-steps of: shooting the subject (7) with a high-definition digital shooting machine (5) with scanning interlaced in a vertical, instead of horizontal, position, on the digital shooting machine (5) oscillating tilt and shift optics being mounted to obtain field depth according to the Scheimpflug rule or condition, which states that, for an optical system, the focal plane generated by the objective and the focal plane of the subject (7) meet on the same straight line; obtaining images of the subject (7) with widespread and uniform illumination on the front to obtain a general opacity of the subject (7) and an illumination of well accurate side cuts and backlights to draw the subject (7) on its whole perimeter to increase the three-dimensionality effect and have well defined edges detached in an empty space, the subject (7) being under total absence of lighting in the areas surrounding his edge, thereby obtaining the “edge cutting” effect necessary for reproducing the images;

– the process further comprises the sub-steps of projecting the image from a LED monitor matrix (26) in a reverse vertical position on a plane made of transparent polycarbonate (28) placed at 45° with respect to the matrix (26);

– the step of reproduction comprises the sub-steps of: providing an optical reproduction system adapted, through the reflection of images generated by a LED monitor matrix (26) or other FULL HD source, to have the optical illusion of the total suspension of an image in an empty space; reflecting (28) the reproduced images on a slab made of transparent polycarbonate; providing a final image perceived by the viewer (1) on a vertical plane (30) behind the polycarbonate slab, completely detached from a background (32), with which a parallax effect (34) is formed; and

– the process comprises the step of activating

the interface for controlling external peripherals outside the optical reproduction system namely domotic peripherals, DMX controlled lights systems for shows or events, a videoprojection of an audio-visual contribution on an external large screen or any other electric or electronic apparatus equipped with standard communication protocols, comprising the steps of:

providing a reproduction (IDLE) of the human being in a standby status;

detecting the presence of one or more users through an input of data provided by the volumetric sensor;

in case of negative input, namely no users are present, going back in a loop to the reproduction of the IDLE status;

in case of positive input, namely users are present, reproducing the PREIDLE sequence and the ACTION sequence with a request to make a control request of the peripheral, possibly showing a list of possible choices;

performing a verbal input by the user;

performing a semantic analysis of the sentence and a logic comparison, possibly executing updated commands by the peripheral;

if the execution is not possible, reproducing the ACTION sequence with a request of inputting again a command on the peripheral;

if the execution is possible, visually executing the BRIDGE sequence and the ACTION sequence in which the interface describes and confirms the execution of the current command and going back to the initial IDLE status;

simultaneously, providing an output through a suitable communication protocol with the peripheral, in which the peripheral is actuated and updated in its status

Is it patentable?

The first-instance Examining Division found that the subject-matter of claim 1 lacked an inventive step over the disclosure of D1 (US 7 844 467) and common general knowledge evidenced by documents D2 (US 2011/007079), D3 (WO 2009/155926), and D5 (US 2011/304632.

Document D1 discloses a user interface for a hand-held device wherein a virtual agent based on computer-generated animated images and voice is presented to the user on the interface’s display. It aims to design the virtual agent as “human” as possible and controls the virtual agent’s facial movements based on stored video recordings of human speakers. The distinguishing features were identified as follows:

2. It was common ground in the oral proceedings that D1 is to be considered, as in the impugned decision, as the closest prior art to the subject-matter of claim 1.

The board agrees with the decision in point 9.1 that at least the following features A to D of claim 1 are not disclosed in D1:

A: – shooting the subject with a shooting machine with scanning interlaced in a vertical (instead of horizontal) position on the digital shooting machine, oscillating tilt and shift optics being mounted to obtain field depth according to the Scheimpflug rule or condition, which states that, for an optical system, the focal plane generated by the objective and the focal plane of the subject meet on the same straight line;

B: – obtaining images of the subject with widespread and uniform illumination on the front to obtain a general opacity of the subject and an illumination of well accurate side cuts and backlights to draw the subject on its whole perimeter to increase the three-dimensionality effect and have well defined edges detached in an empty space, the subject being under total absence of lighting in the areas surrounding its edge, thereby obtaining the “edge cutting” effect necessary for reproducing the images,

– projecting the image from an LED monitor matrix in a reverse vertical position on a plane made of transparent polycarbonate placed at 45° with respect to the matrix, the step of reproduction comprising the sub-steps of:

– providing an optical reproduction system adapted, through the reflection of images generated by an LED monitor matrix or other FULL HD source, to have the optical illusion of the total suspension of an image in an empty space,

– reflecting the reproduced images on a slab made of transparent polycarbonate,

– providing a final image perceived by the viewer on a vertical plane behind the polycarbonate slab,

completely detached from a background, with which a parallax effect is formed;

C: – detecting the presence of one or more users through an input of data provided by the volumetric sensor;

D: – if the execution is not possible, reproducing the ACTION sequence with a request of inputting again a command on the peripheral.

Moreover, the board agrees with the appellant that the following feature is not disclosed in D1:

E: – activating the interface for controlling external peripherals outside the optical reproduction system, namely domotic peripherals, DMX controlled light systems for shows or events, a videoprojection of an audio-visual contribution on an external large screen or any other electric or electronic apparatus equipped with standard communication protocols.

These features included specific technical limitations as can be seen above. The Board then assessed the individual features and acknowledged that they contribute to the inventive step.

In particular, the Board disagreed with the first instance Examining Division and considered that the distinguishing features were not merely common knowledge, and even if it were a common measure, its use was not obvious and its implementation exceeded the general design capabilities of the skilled person:

3. As regards feature A, the decision in point 9.2 states that it improves the quality of the captured image by using the common knowledge of the skilled person, as illustrated inter alia by D3. However, the image of the virtual agent displayed on the interface of D1 is not a captured image of a human actor but rather a computer-generated image based on facial features extracted from video recordings of a human actor. Thus, even if it were to be assumed that feature A in itself was a common measure, the board holds that its use in the system of D1 would not be obvious for the skilled person due to the above-mentioned technical difference between the virtual agent images in D1 and those in claim 1.

4. As regards feature B, the decision in point 9.2 asserts that it only defines how the information is presented to the user and thus has no technical effect other than its pure implementation which is in itself straightforward for the skilled person. However, even if it is considered that obtaining and projecting an image as defined by feature B is an obvious measure and that the image provided by feature B is only a different presentation of a virtual agent to the user with no technical effect, the board holds that implementing the image-obtaining and image-projecting method of feature B in the system of displaying a virtual agent of D1 would necessitate structural modifications of the device supporting the interface of D1 which exceed the general design capabilities of the skilled person.

5. As regards feature E, the board notes that D1 does not hint that the virtual agent may be used as an interface for controlling peripherals which are external to the device supporting the interface, such as those listed in claim 1. The virtual agent is described throughout D1 as a computer-generated image which simulates a person listening and answering to a user of the interface. The only connection of the device supporting the interface with an external entity is a connection with a server (144A in figure 4A and 144B in figure 4B) which assists the device in the task of generating the virtual agent behaviour. None of the other cited documents D2 to D5 relates to the controlling of an external peripheral on the part of an interface simulating a human agent.

6. For these reasons, the board holds that the combination of at least features A, B and E with the prior art of D1 would not be considered by the skilled person without the use of hindsight. Thus, the subject-matter of claim 1 involves an inventive step, having regard to the cited prior art. Claims 2 to 4 are dependent claims and, as such, also meet the requirements of Article 56 EPC.

Therefore, claim 1 was considered to involve an inventive step, and the appeal is allowed with an order to grant the patent.

More information

You can read the whole decision here: T 0799/19 (Cinematographic image interface/BLUE CINEMA) of October 11, 2021 of Technical Board of Appeal 3.5.05

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

www.bardehle.com/de/team/richard-preston

Identification of types of scenes of a sports game: non-technical

This decision relates to a European patent application that concerns the extraction of scenes from a captured video of a sports game. The Board noted that the scenes are defined by characteristics of the audio signal volume based on a heuristic rule reflecting non-technical considerations about the behaviour of spectators at sports games. Thus, the identification of types of scenes of a sports game is not a technical purpose. Here are the practical takeaways from the decision T 2263/18 (Extraction method/FUJITSU) of October 11, 2021, of Technical Board of Appeal 3.5.07:

Key takeaways

The identified scene types correspond to different types of scenes of a sports game and thus relate to the non-technical cognitive content of the video

The definition of the scenes by means of characteristics of the audio signal volume is based on a heuristic rule reflecting non-technical considerations about the behaviour of spectators at sports games.

The fact that the implementation of such a heuristic rule in the system may involve the use of technical means for determining the change amount in the sound volume does not render the underlying heuristic itself technical

The invention

The application relates to the extraction of scenes from a captured video of a sports game. In known video distribution services, it is sometimes desired to identify scenes of interest from a video, for example, scenes in which a point is scored. It was known to recognise a scene of interest by detecting a sharp rise in sound volume in audio data of the video. The present invention proposes to improve the known video extraction techniques for extracting a scene which is defined by a start point and a continuation time (duration of the scene) in a sports video

Fig. 1 of EP 2921973 A1

Claim 1 (main request)

An extraction program (50) that causes a computer (40) to execute processing comprising:

extracting (54) a portion of video data that corresponds to a continuation time as a scene of interest in a sports game, and determining the type of the scene of interest,

wherein the continuation time is a period of time during which a sound volume level of audio data included in footage including audio data and the video data, captured in a sports game, is detected to have a volume greater than a predetermined level indicative of a scene of interest, and

said extracting comprises comparing the continuation time with each of plural thresholds which have different lengths according to a respective type of scene of interest,

the processing further comprising:

detecting, as a start point, a point in time at which the change amount in the sound volume has exceeded a predetermined change amount threshold; and

determining, as the continuation time, a period from the start point in which the sound volume is continually at a level equal to or higher than a specific level based on the sound volume corresponding to the start point.

Is it patentable?

The first-instance Examining Division found that the subject-matter of claim 1 of this request lacked an inventive step over the disclosure of D2 (US 6,160,950) as the distinguishing features were considered as having no synergistic technical effect. The distinguishing feature over document D2 was identified as follows:

A The type of scene of interest is determined by comparing the continuation time against a variety of thresholds, each associated with a respective type of scene of interest.

B The parameter used to set the start point and the continuation time of a scene of interest is the change amount [in the sound volume].

However, the appellant disagreed and argued the features had a technical effect, as follows:

The appellant argued that feature A solved the problem of identifying scene types by comparing the continuation time with each of a set of thresholds corresponding to different scene lengths. These scene lengths defined respective types of scenes of interest. Thus the claimed type determination was based on numerical comparisons, not on any “cognitive interpretation”. The objective technical problem solved by feature A was how to identify scene types, and the solution was not obvious.

Feature B avoided the incorrect extraction of video portions due to incorrectly calibrated audio levels or if the sound level in an arena recorded in sports footage was continuously high. A relative rise in the sound level could be reliably detected by using the “change amount” of the sound level in detecting the start point. Thus using the “change amount” would contribute to accurate detection of the “continuation time” (statement of grounds of appeal, page 8). The extraction method disclosed in document D2 did not address the problems solved.

Finally, features A and B in combination provided the synergistic effect of improving the accuracy of identified scene types, which was a technical effect.

The Board disagreed with the arguments of the appellant, and agreed with the division that the distinguishing feature does not include a synergistic technical effect:

3.3 The board does not recognise the alleged synergistic technical effect. The board agrees that features A and B interact, but the identified scene types correspond to different types of scenes of a sports game (see for example description, paragraph [0021]: goal scenes, disappointment scenes, attack defense switch scenes, foul scenes) and thus relate to the non-technical cognitive content of the video. Hence the combined purpose of features A and B of identifying scene types and their starting points is non-technical.

Moreover, the fact that features A and B interact does not imply that they produce a synergistic effect. Rather, the overall effect is the result of aggregating these features: feature B concerns how the starting point and continuation time are obtained and feature A determines the scene type based on the continuation time. That the scene type is more accurate when the detection of the starting point and the continuation time is more accurate is the expected result of the aggregation of features A and B.

The Board then assessed if the individual features may contribute to the technical purpose that can be used for the assessment of inventive step:

3.4 As the identification of types of scenes of a sports game is not a technical purpose and as the board does not see that feature A involves any “further technical considerations” (see opinion G 0003/08, OJ EPO 2011, 10, Reasons 13.5 and 13.5.1), the board agrees with the examining division that feature A does not contribute to solving a technical problem. Hence it does not enter into the assessment of inventive step (see decision T 154/04, OJ EPO 2008, 46, point 5 (F) of the reasons: “Non-technical features, to the extent that they do not interact with the technical subject matter of the claim for solving a technical problem, i.e. non-technical features ‘as such’, do not provide a technical contribution to the prior art and are thus ignored in assessing novelty and inventive step”).

3.5 The definition of the scenes by means of characteristics of the audio signal volume is based on a heuristic rule reflecting non-technical considerations about the behaviour of spectators at sports games which is usually observed, namely that the sound level increases at time periods of particular interest for the public watching a sports game such as when a goal is scored in soccer. Such time periods then correspond to various types of scenes of interest in the video of the sports game. The fact that the implementation of such a heuristic rule in the system may involve the use of technical means for determining the change amount in the sound volume does not render the underlying heuristic itself technical (see for example decision T 1670/07 of 11 July 2013, reasons 9).

According to the established case law of the boards of appeal, when assessing inventive step in accordance with the problem/solution approach, an aim to be achieved in a non-technical field may legitimately appear in the formulation of the problem as part of the framework of the technical problem to be solved as a constraint that has to be met (see decisions T 641/00, OJ EPO 2003, 352; T 154/04, OJ EPO 2008, 46). Consequently, the heuristic rule may be added as a non-technical constraint to the objective technical problem to be solved.

The board agrees with the examining division that the implementation of the heuristic rule in the program known from document D2 with the aim of determining the type of scenes of interest was straightforward and could not be the basis for acknowledging inventive step. The skilled person was familiar with “change amounts” of sound levels and able to come up with the necessary concepts for implementation.

Therefore, claim 1 was considered not to involve an inventive step, and the appeal is dismissed and the application was refused.

More information

You can read the whole decision here: T 2263/18 (Extraction method/FUJITSU) of October 11, 2021, of Technical Board of Appeal 3.5.07

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

www.bardehle.com/de/team/richard-preston

E-mail read status indication: non-technical

This decision relates to a European patent application that concerns displaying e-mail messages with E-mail read status indication. The Board decided that the rule defining when the flag is to be switched is not based on any technical considerations as it reflects an administrative choice or the user’s subjective preferences. Here are the practical takeaways from the decision T 1227/17 (E-mail read status indication/UNIFY) of November 19, 2021, of Technical Board of Appeal 3.5.01:

Key takeaways

A rule defining when the flag is to be switched is not based on any technical considerations if it reflects an administrative choice or the user’s subjective preferences.

The invention

The invention concerns displaying e-mail messages on a display. E-mail users face the time consuming and sometimes overwhelming task of going through newly received messages

The invention aims at facilitating this task by indicating the read/unread status of each e-mail in the list of displayed e-mails. The status information is changed from “unread” to “read” when the message text and attachments (if present) have been opened and at least partially displayed to the user. In this way, the user can keep track of which e-mail messages have been read. For subsequent e-mails having the same content, the flag is automatically set to “read” when the user opens the first e-mail in the series of e-mails.
Fig. 1 of WO 2014/056516 A1

Claim 1 (main request)

A method for displaying e-mail messages on a display (1) to a user of an e-mail message viewer, the method comprising:

a) displaying a list of received e-mail messages (1) with a list entry (4) for each received email message;

b) displaying binary information indicating the read/unread status of each received email message in the list;

characterized by

c) using a software running on a user’s personal computer or other personal information technology equipment or on a provider’s information technology equipment, which is permanently monitoring texts or more generally contents displayed to the user when reading his received messages and automatically comparing these displayed texts and contents with contents of other e-mails received by the same user;

d) switching the read/unread status information of each received first email message from “unread” to “read” by said software when the text and attachments of this particular first e-mail message have been displayed at least partially to the user when opening this particular first e-mail message;

e) switching the read/unread status information of each received second email message from “unread” to “read” by said software when the text and attachments of this particular second e-mail message have been displayed at least partially to the user when opening a particular first e-mail message.

Is it patentable?

The first-instance Examining Division found that the subject-matter of claim 1 of this request lacked an inventive step over the disclosure of D6 (US 2010/262922).

2.2 D6 discloses, e.g. in paragraph [0029], a method for displaying e-mail messages comprising the step of determining whether the content of an e-mail that has been displayed to the user is found in other e-mails within the same thread. If this is the case, all the e-mails in the thread are marked as “read”.

The distinguishing feature over document D6 was the condition when “read” indication was displayed, and the first instance division considered this as merely an administrative rule:

2.3 It is common ground that the subject-matter of claim 1 of the main request differs from D6 in that the “read” indication for an e-mail is set when both the message body and the attachments have been displayed at least partially to the user.

2.4 The examining division found that this difference was merely an administrative rule defining when an e-mail was to be considered as having been read. The difference did not solve a technical problem, and no further technical effect on the computer system could be established. An inventive step was therefore denied.

However, the appellant disagreed and argued technical effect as follows:

2.5 The appellant argued that the automatic setting or switching of flags by software was a technical feature as it produced a technical action and solved the technical problem of informing the user that an email message has already been read.

The Board disagreed with the arguments of the appellant, f

2.6 However, D6 already discloses the automatic switching of a “read” flag. In other words, the technical problem argued by the appellant is already solved in the prior art. Thus the Board agrees with the division that the invention distinguishes itself from the prior art by the rule defining when the flag is to be switched. This rule is not based on any technical considerations. It rather reflects an administrative choice or the user’s subjective preferences.

2.7 Since the distinguishing feature of claim 1 does not solve a technical problem by providing a technical effect, it does not contribute to inventive step under the “Comvik approach” (see T 641/00 and Case Law of the Boards of Appeal, 9th edition, I.D 9.1.3). Thus, in conclusion, the Board agrees with the examining division that the subject-matter of claim 1 of the main request lacks an inventive step over D6 (Article 56 EPC).

Therefore, claim 1 was considered not to involve an inventive step.

The appellant tried to amend the claim in the Auxiliary Request as ” when the text and attachments of this particular first e-mail message have been displayed at ~~least partially~~ completely to the user.”

However, even such an amendment did not convince the Board which decided that the additional feature of the first auxiliary request concerns the same type of administrative rule as the main request. Thus, it would not contribute to inventive step either.

Therefore the appeal is dismissed and the application was refused.

More information

You can read the whole decision here: T 1227/17 (E-mail read status indication/UNIFY) of November 19, 2021, of Technical Board of Appeal 3.5.01

Stay in the loop

Never miss a beat by subscribing to the email newsletter. Please see our Privacy Policy.

* = Required field

Preston Richard

Key takeaways

The invention

Claim 1 (Sole Request)

Is it patentable?

More information

Stay in the loop

Key takeaways

The invention

Claim 1 (Main Request)

Claim 1 (Second Auxiliary Request)

Is it patentable?

Claim 1 (Fourth Auxiliary Request)

More information

Stay in the loop

Key takeaways

The invention

Claim 1

Is it patentable?

More information

Stay in the loop

Key takeaways

The invention

Claim 1 of the Main Request

The Referral (G 1/19)

Is it patentable?

Claim 1 of the Fourth Auxiliary Request - amendments emphasised in Bold

Claim 1 of the Seventh Auxiliary Request - amendments emphasised in Bold

More information

Stay in the loop

Key takeaways

The invention

Claim 1 of the Main Request

Is it patentable?

More information

Stay in the loop

Key takeaways

The invention

Claim 1 (Mobile Device) of the Sole Request

Is it patentable?

More information

Stay in the loop

Key takeaways

The invention

Claim 1

Is it patentable?

More information

Stay in the loop

Key takeaways

The invention

Claim 1 (main request)

Is it patentable?

More information

Stay in the loop

Key takeaways

The invention

Claim 1 (main request)

Is it patentable?

More information

Stay in the loop

Key takeaways

The invention

Claim 1 (main request)

Is it patentable?

More information

Stay in the loop