Navigating network-based electronic information using spoken input with multimodal error feedback
DC CAFC
First Claim
Patent Images
1. A method for speech-based navigation of an electronic data source, the electronic data source being located at one or more network servers located remotely from a user, comprising the steps of:
- (a) receiving a spoken request for desired information from the user;
(b) rendering an interpretation of the spoken request;
(c) constructing at least part of a navigation query based upon the interpretation;
(d) soliciting additional input from the user, including user interaction in a non-spoken modality different than the original request without requiring the user to request said non-spoken modality;
(e) refining the navigation query, based upon the additional input;
(f) using the refined navigation query to select a portion of the electronic data source; and
(g) transmitting the selected portion of the electronic data source from the network server to a client device of the user.
2 Assignments
Litigations
10 Petitions
Accused Products
Abstract
A system, method, and article of manufacture are provided for navigating an electronic data source by means of spoken language. When a spoken input request is received from a user, it is interpreted. Additional input is solicited from the user in a modality different than the original request and used to refine the navigation query. The resulting interpretation of the request is thereupon used to automatically construct an operational navigation query to retrieve the desired information from one or more electronic network data sources.
484 Citations
130 Claims
-
1. A method for speech-based navigation of an electronic data source, the electronic data source being located at one or more network servers located remotely from a user, comprising the steps of:
-
(a) receiving a spoken request for desired information from the user;
(b) rendering an interpretation of the spoken request;
(c) constructing at least part of a navigation query based upon the interpretation;
(d) soliciting additional input from the user, including user interaction in a non-spoken modality different than the original request without requiring the user to request said non-spoken modality;
(e) refining the navigation query, based upon the additional input;
(f) using the refined navigation query to select a portion of the electronic data source; and
(g) transmitting the selected portion of the electronic data source from the network server to a client device of the user. - View Dependent Claims (2, 4, 5, 6, 7, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24)
3.The method of claim 1, wherein the step of constructing a navigation query further includes the steps of extracting an input template for an online scripted interface to the data source, and using the input template to construct the navigation query. -
4. The method of claim 1, wherein the navigation query is constructed in the format of a database query language.
-
5. The method of claim 1, wherein the step of rendering an interpretation and the step of constructing a navigation query are performed, at least in part, on a computing device located locally with the user.
-
6. The method of claim 1, wherein the step of rendering an interpretation and the step of constructing a navigation query are performed, at least in part, on a network computing device located remotely from the user.
-
7. The method of claim 1, wherein the step of soliciting additional input is performed in response to one or more deficiencies encountered during the step of constructing a navigation query.
-
9. The method of claim 8, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken request.
11.The method of claim 1, wherein the step of soliciting additional input is performed in response to one or more deficiencies encountered after a first navigation of the data source using the navigation query constructed in step (c). -
12. The method of claim 1, wherein the additional input is solicited upon receiving a user-input statement that additional information is required.
-
13. The method of claim 1, wherein the step of soliciting the additional input includes presenting a menu to the user on the client device of the user.
-
14. The method of claim 1, wherein the step of soliciting the additional input includes presenting a textual request for the additional input.
-
15. The method of claim 1, wherein the step of soliciting the additional input includes an audible request for the additional input.
-
16. The method of claim 1, wherein the step of soliciting the additional input includes presenting a list of portions of the electronic data source that match the navigational query.
-
17. The method of claim 1, wherein additional input received from the user is at least partially speech based.
-
18. The method of claim 1, wherein additional input received from the user includes no spoken input.
-
19. The method of claim 1, wherein steps (d)-(e) are repeated until the navigational query is deemed adequate.
-
20. The method of claim 1, wherein the input modality of step (d) includes selecting from a displayed option menu.
-
21. The method of claim 22, wherein the act of selecting from the displayed option menu is performed by speaking.
-
22. The method of claim 1, wherein the method is performed with respect to a plurality of simultaneous users and corresponding client devices.
-
23. The method of claim 1, further including the step of selecting the data source from among a plurality of candidate electronic data sources, in response to the interpretation of the spoken request.
-
24. The method of claim 1, wherein the electronic data source stores multimedia content including at least one of video content and audio content.
-
-
3. The method of claim 3, wherein the step of extracting the input template includes dynamically scraping the online scripted interface.
-
8. The method of claim 8, wherein the deficiencies include unresolved words of the spoken request.
- 11. The method of claim 11, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.
-
25. A system for speech-based navigation of an electronic data source, the electronic data source being located at one or more network servers located remotely from a user, the system comprising:
-
(a) a portable microphone operable to receive a spoken request for desired information from the user;
(b) language processing logic, operable to render an interpretation of the spoken request;
(c) query construction logic, operable to construct a navigation query in response to the interpretation of the spoken request;
(d) user interaction logic, operable to solicit additional input from the user, including user interaction in a non-spoken modality different than the original request without requiring the user to request said non-spoken modality;
(e) query refining logic, operable to refine the navigation query, based upon the additional input;
(f) navigation logic, operable to select a portion of the electronic data source using the navigation query; and
(g) electronic communications infrastructure for transmitting the selected portion of the electronic data source from the network server to a primarily stationary, display device located locally with the user.
-
- 27. The system of claim 27, wherein the language processing logic extracts an input template for an online scripted interface to the data source, and uses the input template to construct the navigation query.
- 34. The system of claim 34, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken request.
-
44. A computer program embodied on a computer readable medium for speech-based navigation of an electronic data source, the electronic data source being located at one or more network servers located remotely from a user, comprising:
-
(a) a code segment that receives a spoken request for desired information from the user;
(b) a code segment that renders an interpretation of the spoken request;
(c) a code segment that constructs at least part of a navigation query based upon the interpretation;
(d) a code segment that solicits additional input from the user, including user interaction in a non-spoken modality different than the original request without requiring the user to request said non-spoken modality;
(e) a code segment that refines the navigation query, based upon the additional input;
(f) a code segment that uses the refined navigation query to select a portion of the electronic data source; and
(g) a code segment that transmits the selected portions of the electronic data source from the network server to a primarily stationary, display device located locally with the user.
-
- 46. The computer program of claim 46, further comprising a code segment that extract an input template for an online scripted interface to the data source, and a code segment that uses the input template to construct the navigation query.
- 53. The computer program of claim 53, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken request.
-
70. A method for utilizing spoken natural language for navigating an electronic data source, the electronic data source being located at one or more network servers located remotely from a user;
- comprising the steps of;
(a) receiving a spoken natural language (“
NL”
) request for desired information from the user;
(b) rendering an interpretation of the spoken request;
(c) constructing at least part of a navigation query based upon the interpretation;
(d) soliciting additional input from the user, including user interaction in a non-spoken modality different than the original request without requiring the user to request said non-spoken modality;
(e) refining the navigation query, based upon the additional input;
(f) using the refined navigation query to select a portion of the electronic data source; and
(g) transmitting the selected portion of the electronic data source from the network server to a client device,of the user.
- comprising the steps of;
- 72. The method of claim 72, wherein the step of constructing a navigation query further includes the steps of extracting an input template for an online scripted interface to the data source, and using the input template to construct the navigation query.
- 79. The method of claim 79, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken NL request.
- 82. The method of claim 82, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.
-
88. A system or utilizing spoken natural language to navigate an electronic data source, the electronic data source being located at one or more network servers located remotely from a user, the system comprising:
-
(a) a portable microphone operable to receive a spoken natural language (“
NL”
) request for desired information from the user;
(b) spoken language processing logic, operable to render an interpretation of the spoken natural language request;
(c) query construction logic, operable to construct a navigation query in response to the interpretation of the spoken natural language request;
(d) user interaction logic, operable to solicit additional input from the user, including user interaction in a non-spoken modality different than the original request without requiring the user to request said non-spoken modality;
(e) query refining logic, operable to refine the navigation query, based upon the additional input;
(f) navigation logic, operable to select a portion of the electronic data source using the navigation query; and
(g) electronic communications infrastructure for transmitting the selected portion of the electronic data source from the network server to a primarily stationary, display device located locally with the user.
-
- 90. The system of claim 90, wherein the spoken language processing logic extracts an input template for an online scripted interface to the data source, and uses the input template to construct the navigation query.
- 97. The system of claim 97, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken NL request.
- 100. The system of claim 100, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.
-
107. A computer program embodied on a computer readable medium for utilizing spoken natural language for navigating an electronic data source, the electronic data source being located at one or more network servers located remotely from a user, comprising:
-
(a) a code segment that receives a spoken natural language (“
NL”
) request for desired information from the user;
(b) a code segment that renders an interpretation of the spoken natural language request, (c) a code segment that constructs at least part of a navigation query based upon the interpretation;
(d) a code segment that solicits additional input from the user, including user interaction in a non-spoken modality different than the original request without requiring the user to request said non-spoken modality;
(e) a code segment that refines the navigation query, based upon the additional inputs;
(f) a code segment that uses the refined navigation query to select a portion of the electronic data source; and
(g) a code segment that transmits the selected portion of the electronic data source from the network server to a primarily stationary, display device located locally with the user.
-
- 109. The computer program of claim 109, further comprising a code segment that extract an input template for an online scripted interface to the data source, and a code segment that uses the input template to construct the navigation query.
- 116. The computer program of claim 116, wherein the deficiencies include one or more required elements of the navigational query not determinable from the interpretation of the spoken NL request.
- 119. The computer program of claim 119, wherein the deficiencies include failure to identify a single data record within the data source responsive to the navigation query.
-
125. A method for utilizing spoken natural language for navigating an electronic data source, the electronic data source being located at one or more network servers located remotely from a user, comprising the steps of:
-
(a) receiving a spoken natural language (“
NL”
) request for desired information from the user;
(b) rendering an interpretation of the spoken request;
(c) constructing at least part of a navigation query based upon the interpretation;
(d) soliciting additional input from the user, including user interaction in a non-spoken modality different than the original request, in accordance with results generated from said at least part of a navigation query;
(e) refining the navigation query, based upon the additional input;
(f) using the refined navigation query to select a portion of the electronic data source; and
(g) transmitting the selected portion of the electronic data source from the network server to a client device of the user.
-
-
128. A method for utilizing spoken natural language for navigating an electronic data source, the electronic data source being located at one or more network servers located remotely from a user, comprising the steps of:
-
(a) receiving a spoken natural language (“
NL”
) request for desired information from the user;
(b) rendering an interpretation of the spoken request;
(c) constructing at least part of a navigation query based upon the interpretation;
(d) soliciting additional input from the user, including user interaction in a non-spoken modality different than the original request, in response to one or more deficiencies encountered during the step of constructing said at least part of a navigation query;
(e) refining the navigation query, based upon the additional input;
(f) using the refined navigation query to select a portion of the electronic data source; and
(g) transmitting the selected portion of the electronic data source from the network server to a client device of the user. - View Dependent Claims (126, 127)
-
- 130. The method of claim 131, wherein the act of selecting from the displayed option menu is performed by speaking.
Specification
-
- Resources
Litigation Campaign Assessment
Thank you for your request. You will receive a custom alert email when the Litigation Campaign Assessment is available.
×
Litigation Data
-
Current AssigneeIPA Technologies Inc. (Quarterhill Inc. (f/k/a Wi-LAN Inc.))
-
Original AssigneeSRI International, Inc.
-
InventorsHalverson, Christine, Julia, Luc, Voutsas, Dimitris, Cheyer, Aden J.
-
Primary Examiner(s)Trammell, James P.
-
Assistant Examiner(s)Backer, Firmin
-
Application NumberUS09/524,095Time in Patent Office1,534 DaysField of Search709/218, 707/5, 707/4, 707/102, 704/257, 704/231US Class Current709/218CPC Class CodesG06F 16/3334 Selection or weighting of t...G06F 16/73 QueryingG06F 16/90332 Natural language query form...G06F 9/465 Distributed object oriented...G06N 5/043 Distributed expert systems;...G10L 2015/223 Execution procedure of a sp...H04M 2201/40 using speech recognition sp...H04M 3/4936 Speech interaction details ...H04M 3/4938 comprising a voice browser ...Y10S 707/99934 Query formulation, input pr...Y10S 707/99935 Query augmenting and refini...Y10S 707/99943 Generating database or data...
