Method and apparatus for providing a more powerful user

Method and apparatus for providing a more powerful user

US007577910B1

(12) United States Patent

Husemann et a].

(10) Patent N0.:

(45) Date of Patent:

US 7,577,910 B1

Aug. 18,2009

LIMITED USER-INTERFACE

(75) Inventors: Dirk Husemann, Adliswill (CH);

Michael Moser, Zurich (CH)

(73) Assignee: International Business Machines

Corporation, Armonk, NY (US)

( * ) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35

U.S.C. 154(b) by 1343 days.

(21) App1.N0.: 09/613,113

(22) Filed: Jul. 10, 2000

(30)

Jul. 12, 1999

Foreign Application Priority Data

(EP) ................................ .. 99113414

(51) Int. Cl.

G06F 3/00 (2006.01)

(52) US. Cl. ..................... .. 715/744; 715/745; 715/746;

715/747

(58) Field of Classi?cation Search ............... .. 345/716,

345/719, 720, 721, 723, 730; 7l5/744i749

See application ?le for complete search history.

(56) References Cited

U.S. PATENT DOCUMENTS

6,173,316 B1 * 1/2001 De Boor et al. ........... .. 709/218

6,216,158 B1 * 4/2001 Luo et al. . . . . . . . . . . .. 709/217

6,446,096 B1 * 9/2002 Holland et al. ............ .. 715/513

6,456,892 B1 * 9/2002 Dara-Abrams et al. ...... .. 700/83

6,466,971 B1 * 10/2002 Humpleman et al. ...... .. 709/220

6,502,000 B1 * 12/2002 Arnold et al. ............... .. 700/83

6,509,913 B2* 1/2003 Martin et al. 345/762

6,560,640 B2 * 5/2003 Smethers .................. .. 709/219

JP

JP

JP

JP

JP

JP

JP

W0

W0

6,792,605 B1 * 9/2004 Roberts et al. ............ .. 719/313

FOREIGN PATENT DOCUMENTS

07-254920 l0-l62060 l0-240833

11-027290

11-031114 ll-l6l32l

2002-509669 l0/l995

6/1998

9/1998 l/l999

2/1999

6/1999

3/2002 l0/l995 l2/l998

European Search Report, Application No. EP 99 111 3414, Mar. 21,

2000.

J aap Haartsen, “BluetoothiThe universal radio interface for ad hoc, wireless connectivity”, Ericsson Review No. 3, 1998.

Christer Erlandson and Per Ocklind, “WAPiThe wireless applica tion protocol”, Ericsson Review No. 4, 1998.

* cited by examiner

Primary ExamineriBa Huynh

(74) Attorney, Agent, or FirmiBrian P. Verminski, Esq.

(57) ABSTRACT

with a limited user-interface via a remote computer device having a more powerful user interface. Both computer devices are interconnected via a wireless communication channel and both computer devices support a common com munications protocol. User-interface information is sent from the computer device with a limited user-interface to the remote computer device to provide a user-interface at the remote computer device for receiving user input at the remote computer device. The user input controls the computer device with a limited user-interface to execute commands therein corresponding to the user input.

26 Claims, 10 Drawing Sheets

background

receive service information

103 user requests (list of) devices send user input to device

US. Patent Aug. 18, 2009 Sheet 1 0f 10 US 7,577,910 B1

16

brought

into vicinity

I/deIauIt-IontsI

FIG. 1

Vlmer'aw

[/lorrhéts

F

I

I

PJCL.

[/PostScript I

I/baud-raté]

'I/parity

FIG. 2

US. Patent

Aug. 18,2009

Sheet 2 0f 10

US 7,577,910 B1

32~

I | '

15E

' blueloolhJIsony_cdp_990)<.lmain_merlu.wml

12

,36

/

_,

33

General settings

Welcome to

SONY CD-Player 990x

Please select the function you want lo

con?gure:

‘ CO_names

help |"“’'37

35~

. CD labels

3.

30

FIG. 3

US. Patent Aug. 18, 2009 Sheet 3 0f 10 US 7,577,910 B1

URL: uh'?sony_cdp_990)<!main_menu.wml#CD_naes ,35

Enter a name for the currently inserted

CD1

1 i;

!

Generalsmings ,/ co-mm“ _ 34 play__rnode

help :1

_/'5'7

/""_’4O

FIG. 4

US. Patent Aug. 18, 2009 Sheet 4 of 10 US 7,577,910 B1

51

31

K,

(A e‘. ‘M ; ,. ..‘_, _ E j

éURL: bluetoolhJlsony__cdp_Q90>(Imain_menu.wm|#play

2 _ -

Play modes.

Select one ofthe following play cgmms phyjnode /}

,_../”34

Mp ~13? "play-modes:

6) Normal

(3 Random

O Shu?le

C- User-De?ned

‘50

5M

U

52

J

FIG. 5

US. Patent Aug. 18, 2009 Sheet 5 0f 10 US 7,577,910 B1

FIG. 6

US. Patent Aug. 18, 2009 Sheet 6 0f 10 US 7,577,910 B1

user interface

manager

I

[72

MAC protocol

[73 transmitter

0

_L. driver

2 receiver driver

\74

[79

application

[a

HW-driver(s) 1Q device speci?c

Fli'L

2i

55:28.

70/

B232

3 1i

2: DOE

I

L

2f

386:: 3: MT

FIG. 7A

US. Patent Aug. 18, 2009 Sheet 7 0f 10 US 7,577,910 B1

I

[720

MAC

protocol

[730

transmitter driver

receiver

driver

Um

nSoa

.nm. ng mu

rn

C l 0

f

750

user interface drivcr

(IL

55:22.

7€l 1%)2

8232 0 v

700/

03 7Y4 boEuE 0 r

EU“.

385:: .55 0 7

0

FIG. 7B

00

FIG. 7C

US. Patent Aug. 18, 2009 Sheet 8 0f 10 US 7,577,910 B1

' background

A

8761

H . . - - - - - "

. .

W

'

information

L“ ._

I

_______

I ' _ _ Mwfer. 87

receive input & {-88 categorize cond. update own US! with services

request to send UI

Send UI description

7 \

35

/

handle/execute user input

I . . . . . . . .

[90 send feedback

FIG. 8

US. Patent Aug. 18, 2009 Sheet 9 0f 10 US 7,577,910 B1

/02

‘'3 1/103 receive service I information I‘

background

91

user requests (list of) devices

I

/92

display list of controllable devices

user selects device

—93

a

send request to send [II-description

_ /-—96

receive UI-descn'ption

Y render UI-description

(e.g. display or read to user)

['6CCIVC USCI' input

/

_

US 7,577,910 B1

1 2

LIMITED USER-INTERFACE

TECHNICAL FIELD cation feature (such as the Casio PC Unite Data Bank Watch,

HBX-lOOB-l) used to connect to a PC.

There are many other examples of user-interfaces that are severely lacking for various reasons, the most prominent of

Which are siZe and cost constraints. Often such user-interface restrictions make the respective devices less useful for their oWners than they could be.

The present invention relates to computer systems Which have a limited user-interface, such as hand-held pervasive computing devices, and in particular to a design for allowing easy interaction With such computer systems.

It Would thus be useful to have a Way to unleash the full potential of all these devices and to program and con?gure them much more conveniently, thereby making them more useful to their users.

There is groWing demand in the industry to offer devices

BACKGROUND OF THE INVENTION that are ‘open’ in the sense that a user has access via an interface to the device’s processor or other components. An

Through relatively recent technological innovations, com puter systems have become dramatically smaller and more ideal ‘open’ device Would be fully controlled by the user, preferably Within Well-de?ned rules to prevent misuse or

portable. Even very poWerful personal computers (PCs), for

destruction of the device itself. example, are small enough to sit on the desk at Work. Smaller still are lap top computers and notebook computers. There are computer terminals Which are small enough to be mounted in

In addition, there are a groWing number of devices that are netWork enabled, Which means that they can communicate

20

With one or more other devices via a netWork. This can be a vehicle, such as a delivery truck. Still smaller are hand held terminals, Which are typically used for their portability fea tures, alloWing a user to carry the terminal in one hand and operate it With the other. achieved using physical connections, such as cables or ?bers, for example. As these devices get smaller, hoWever, it becomes desirable to replace the physical connections With

Wireless connections (e.g. body netWorks, radio frequency

In addition, there is a trend toWard offering consumers

25 electronic devices that include some sort of computer system, e.g., a microprocessor. Usually, these computer systems not only control the operation or function of the consumer device, connections, or infrared connections), since physically con necting the devices by means of cables or ?bers severely

reduces the ef?ciency gained by making the units smaller.

Ad-hoc Wireless connections are required Where devices but also provide some interface for a user or operator to control certain functions or parameters according to actual move around, enter an area and exit the area. The term ad-hoc

30 refers to the need for frequent netWork reorganiZation. needs. It is in the nature of these consumer devices that they

In addition, there are many different knoWn communica do not have a full user-interface like a computer With display and keyboard. It is not likely that a dishWasher, for example, tions protocols or standards that have been developed and

designed (and continue to be developed and designed)

Will ever have such a full user-interface. In some cases the interface is limited due to space constraints (a typical example

35 for this is a Wrist Watch), While in other cases the interface is limited to keep the cost of manufacturing loW, While still in directed at communication betWeen devices or subsystems.

HereinbeloW, some Wireless communications protocols or standards Will be mentioned. There are many ?ber or cable based, standardized approaches that are suited for such com other cases the processing poWer of the computer system, or the constrained memory space, limits the interaction betWeen the user and system.

40 munication as Well.

GTE Corporation has developed a short-range radio-fre quency (RF) technique Which is aimed at giving mobile devices such as cellular phones, pagers and hand-held per

A typical example is a compact disk (CD) player Which alloWs programming of CD titles using a small four button sonal computers (PCs) a smart Way to interact With one another. GTE’s technique is tentatively named Body LAN control. Programming of such a CD player is very cumber

(local area netWork). The original development of Body LAN

some because one needs to use the buttons to move through

45

Was via a Wired vest With Which various devices Were con the entire alphabet to select letters and/or numbers. Another example is a Wrist Watch that alloWs the user to enter phone book entries, appointments, and to-do items. Typically, the keyboard includes a very limited number of keys. Further nected (hence the name Body LAN). This then developed into to an RF connection.

Xerox Corporation has developed a hand-held computing device called PARC TAB. The PARC TAB is portable yet more, the display is small and its resolution limited. Certain

50 keys have to be pressed several times to reach special char connected to the of?ce Workstation through base stations

Which have knoWn locations. The PARC TAB base stations are placed around the building, and Wired into a ?xed Wired acters, or to activate special functions. Yet another example is a personal digital assistant (PDA) With a touch sensitive screen. In this case the screen occupies most of the device’s netWork. The PARC TAB system uses a preset knoWledge of the building layout and the identi?ers of the various base surface and there are very feW buttons, if any. Some functions are easily accessible using a pointing device, but other func tions have to be selected or activated ?ipping through several

55 layers of menus, for example. Other examples are telephones, vending machines, microWave ovens, mobile phones, etc. For stations to determine the strongest base station signal for a

PARC TAB portable device. A PARC TAB portable device has a Wireless interface to the base stations. The PARC TAB system assumes that the PARC TAB portable device is alWays connected to the netWork infrastructure. The location of each the purposes of the present description these devices are

60 referred to hereinafter as user-interface limited devices. portable PARC TAB device is alWays knoWn to the system softWare. The base stations establish regions and are con

Currently there are a feW approaches using a personal computer (PC) to run better user-interfaces, eg the “Nokia

Cellular Data Suite” for mobile phones alloWs the entry of phone book data. The Cellular Data Suite is a hardWare and

65 softWare package from Nokia designed for cellular phones.

Another example is a Wrist-Watch that has an IR-communi nected to poWer supplies. PARC TAB communication sys tems have a star topology.

In an attempt to standardiZe data communication betWeen

disparate PC devices, several companies, including Ericsson,

IBM, Intel, Nokia, and Toshiba have established a consortium to create a global standard for Wireless RF-based connectivity

US 7,577,910 B1

3

4

between ?xed, portable and mobile devices. There are many other companies adopting the proposed standard. The pro posed standard is called Bluetooth and comprises architecture ing Group’s web site http://www.homerf.org. The SWAP

speci?cation 1.0 is incorporated by reference in its entirety.

There are several other known protocols and techniques

and protocol speci?cations ranging from the physical layer up

to the application layer. The Bluetooth standard contemplates that allow communication between two or more devices. The above described Bluetooth radio technology and HomeRF approach are prominent wireless examples. allowing users to connect a wide range of devices easily and quickly, without the need for cables, expanding communica tions capabilities for mobile computers, mobile phones and other mobile devices. The Bluetooth operating environment is not yet fully de?ned, but similarities are expected with the

It is therefore an object of the present invention to provide

IrDA (Infrared Data Association) speci?cation and the

Advanced Infrared (AIr) speci?cation. It is not unreasonable a method and apparatus for providing a more powerful user interface to a device with a limited user-interface. to expect that the Bluetooth standard will eventually incorpo rate aspects of the IEEE standard 802.1 1 and/ or HIPERLAN,

It is another object of the present invention to provide a method and apparatus for simpli?ed and improved user inter as promulgated by the European Telecommunications Stan action with a device with a limited user-interface, also dards Institute (ETSI).

Bluetooth radio technology provides a standard protocol referred to herein as an “interface limited device.”

To achieve the above objects, a method and apparatus in suitable for forming small private ad-hoc groupings of con accordance with the present invention includes a standard nected devices away from ?xed network infrastructures. wireless communications protocol which allows a user to

Bluetooth makes a distinction between a master unitiwhich

20 interact with or control an interface limited device wirelessly is a device whose clock and hopping sequence are used to using a second (independent) device. synchronize all other devicesiand slave units in the same

In other words, the present invention provides a more pow network segment. In other words, the Bluetooth approach is erful user-interface to an interface limited device by interfac ing it with a more powerful device in its vicinity. The more ing Bluetooth devices with an unknown address. Queries are

25 extensive input capabilities of the more powerful device (sec also centraliZed at a registry server. Further details can be ond device) are employed to control certain aspects of the

found in Haartsen, Allen, Inouye, Joeressen, and Naghshineh,

“Bluetooth: Vision, Goals, and Architecture” in the Mobile

Computing and Communications Review, Vol. 1, No. 2.

Mobile Computing and Communications Review is a publi

limited user-interface device (?rst device).

The method in accordance with the present invention includes the steps of transmitting user-interface information

30 from the ?rst device to the second device; providing a user interface at the second device corresponding to the user

HomeRF (based on Shared Wireless Access Protocol interface information; receiving user input via the user-inter

(SWAP)) is another example of a prospective operating envi ronment protocol which can be used to connect devices. A

HomeRF Working Group was formed to provide the founda tion for a broad range of interoperable consumer devices by

35 establishing an open industry speci?cation for wireless digi face at the second device; transmitting user command information corresponding to the user input from the second device to the ?rst; and executing the corresponding user com mands at the ?rst device. tal communication between PCs and consumer electronic devices anywhere in and around the home. The working

DESCRIPTION OF THE DRAWINGS group, which includes the leading companies from the per

40

The above and other objects, features and advantages of the sonal computer, consumer electronics, peripherals, commu present invention will become more apparent in light of the nications, software, and semiconductor industries, is cur rently developing a speci?cation for wireless following detailed description of an exemplary embodiment thereof taken in conjunction with the attached drawings in communications in the home called the SWAP. The HomeRF which:

SWAP system is designed to carry both voice and data tra?ic and to interoperate with the Public Switched Telephone Net

45

FIG. 1 is a block diagram illustrating a typical environment work (PSTN) and the Internet. It operates in the 2400 MHZ band and uses a digital frequency hopping spread spectrum wherein the present invention is utiliZed, in accordance with an embodiment of the present invention;

FIG. 2 is an illustrative representation of an exemplary

radio. The SWAP technology protocol being developed is

execution tree in accordance with an embodiment of the being derived to some extent from extensions of existing

50

present invention;

FIG. 3 illustrates a ?rst user-interface window in accor enable a new class of home cordless services. It envisions supporting both a time division multiple access (TDMA) service to provide delivery of interactive voice and other dance with an embodiment of the present invention;

FIG. 4 illustrates a second user-interface window in accor dance with an embodiment of the present invention; time-critical services, and a carrier sense multiple access/

55 collision avoidance (CSMA/CA) service for delivery of high speed packet data. The SWAP system is contemplated to

FIG. 5 illustrates a third user-interface window in accor dance with an embodiment of the present invention; operate either as an ad-hoc network or as a managed network

FIG. 6 is a block diagram illustrating another embodiment of a typical environment in which the present invention is under the control of a connection point. In an ad-hoc network, where only data communication is supported, all stations will be equal and control of the network will be distributed between stations. For time critical communications such as

60 utiliZed, in accordance with an embodiment of the present

invention;

FIG. 7A is a block diagram illustrating a ?rst device (with interactive voice, the connection pointiwhich provides the gateway to the PSTNiwill be required to coordinate the

65 a limited user-interface) in accordance with an embodiment of the present invention;

FIG. 7B is a block diagram illustrating a second device a connection point and other stations. Further details about

HomeRF can be found at the Home Radio Frequency Work with a more powerful user-interface) in accordance with an embodiment of the present invention;

US 7,577,910 B1

5

6

FIG. 7C is a block diagram illustrating the hardware layer of a second device in accordance with an embodiment of the addressing design is also required in a GSM-based imple mentation of the present invention.

present invention;

FIG. 7D is a block diagram illustrating the hardware layer of a ?rst device in accordance with an embodiment of the

present invention;

It is understood by those skilled in the art that at the present time many of the protocols that are suited for use in wireless communications systems are still in draft status. The present design is independent of any one particular protocol and can

FIG. 8 is a ?owchart illustrating the method of the present invention in a ?rst device, in accordance with an embodiment be used in connection with many such protocols. Accord ingly, someone having ordinary skill in the art is able to

implement the present design in existing protocol environ

of the present invention;

FIG. 9 is ?owchart illustrating the method of the present ments as well as in protocol environments under development invention in a second device to control a ?rst device in accor or yet to be developed.

The present design can be used anywhere inside, i.e. ware dance with an embodiment of the present invention; and

FIG. 10 is a Wireless Markup Language (WML) document describing the user-interfaces of FIGS. 3-5. houses, on manufacturing ?oors, in o?ices, on trading ?oors, in private homes, and outside of buildings, in cars and trucks, in airplanes, just to mention a few examples.

Two devices can be connected using a 1:1 connection.

DESCRIPTION OF PREFERRED

Possible media are infrared and magnetic ?elds. The proce

EMBODIMENTS dure to setup such a 1:1 connection can be similar to today’s setup of a connection between two IrDA enabled devices, i.e.

For the purpose of the present description, a network can be

20 anything that allows a ?rst device (the limited user-interface the devices must be positioned such that their communication subsystems (transceivers) can “see” each other. Then both device) to communicate with a second device (which has a systems are triggered to start a connection setup procedure more powerful user-interface).A simple point-to-point link, a until a wireless communication channel is established.

Likewise, two devices can be connected using a shared link or any other kind of link is hereinafter referred to as network. This network can either be a physical network or a

25

Possible systems could be based on technology and protocols

wireless network (e.g., infrared (IR), radio-frequency (RF),

such as HomeRF). The network may be completely isolated from any other network, or it might comprise one or more

like BlueTooth, DECT, and HummingBird.

Details about HummingBird transceivers are given in

“Hummingbird Spread Spectrum Transceiver Operator’s

access points which provide the devices with access to another network.

30

The speci?c range that constitutes a wireless network in accordance with the present invention depends on actual implementation details. Generally, a wireless network can be

Manual”, Rev. 24 Jun. 1998, XETRON Corp., Cincinnati,

Ohio, USA

Details concerning the basic problems regarding identi? cation and addressing, initial (resource) discovery, matching

described as having a coverage area between a few square

35 meters and several thousands of square kilometers (e.g., in and selection of communication partners, etc. depend on the medium used and the communications protocol employed. case of a GSM network). Under certain circumstances the can establish a network connection to another device. communication range may go even further. The two

Examples of devices are: laptop computers, workpads, node devicesiwhich communicate with each other have to be “in vicinity,” which means that they have to be suf?ciently close or otherwise interconnected in order to be able to exchange

40

pads, personal digital assistants (PDAs), notebook computers

and other wearable computers, desktop computers, computer terminals, networked computers, internet terminals and other information with each other.

computing systems, set-top boxes, cash registers, bar code

The devices need to be able to transmit and/or receive scanners, point of sales terminals, kiosk systems, cellular information via the network. Accordingly, two devices that

phones, pagers, wrist watches, digital watches, badges, and

communicate with each other must support the same commu

45 smart cards. Other contemplated devices include: headsets,

nication protocol.

Human Interface Device (HID) compliant peripherals, data

Well suited to support communication between devices is the Bluetooth communications design, which is described in

the Haartsen, Allen, Inouye, Joeressen, and Naghshineh,

and voice access points, cameras, printers, fax machines,

keyboards, joysticks, HiFi systems, audio (sound) cards, loudspeakers, ampli?ers, video cards, kitchen appliances,

“Bluetooth: Vision, Goals, and Architecture” in the Mobile

Computing and Communications Review, Vol. 1, No. 2.

Mobile Computing and Communications Review is a publi

50 tools, sensors such as smoke and/or ?re detectors, and virtu ally any other digital device.

Other examples of devices that can be used in connection with the present invention are, personal effects being rated herein by reference in its entirety.

It is assumed, that once the devices are in vicinity of each

55 other, a wireless communication path between these devices can be established4e.g. using magnetic ?eld (near ?eld/5-30 equipped with computer-like hardware, such as a “smart wal let” computer, jewelry, or articles of clothing. In addition to a

“smart wallet” computer, there are a number of other varia tions of wearable computers. A “belt” computer is such a cm), infrared (IR), e.g., IrDA (0.5-2 m) or AIr (1-10 m), or

low-power radio-frequency (RF) communication, e.g. Blue

Tooth (71-10 m), or HomeRF (71-50 m), just to list some

60 examples of wireless schemes that are suited. variation which allows the user to surf, dictate, and edit docu ments while they are moving around. Yet another example is a child’s computer which is comparable to a personal digital assistant for grade-school children. The child’s computer

might hold assignments, perform calculations, and help kids

plished via some global addressing design and a local proxy

(e. g. an IR-beacon on each ceiling or a BlueTooth “relay” in manage their homework. It can interface with other children’ s computers to facilitate collaboration, and it can access a teacher’s computer to download assignments or feedback. each room or home) connected to a network (eg the Inter net), thus effectively allowing remote control of a variety of

65 devices from virtually anywhere in the world. Such a global

Any wearable or portable device, any of?ce tool or equip ment, home tool or equipment, system for use in vehicles, or

US 7,577,910 B1

7

8

systems for use in the public (i.e. vending machines, ticketing

machines, automated teller machines, etc.) may include the present invention.

It is also assumed that a device, as used in connection With the present invention, includes a minimum amount of pro cessing poWer to enable it to participate in the design accord ing to the present invention. These devices are thus also

?rst device 10 by transmitting a more poWerful user-interface

(user-interface description 15) stored in the ?rst device 10 to the second device 12. The ?rst device 10 and second device 12 communicate via a Wireless communications channel 16. A more poWerful user-interface is a user-interface that is easier referred to as computer devices. Most, if not all, of the above listed devices may be vieWed as being devices With limited user-interfaces. This may even be the case for a personal computer Which has a display and a keyboard. There is still room for improvement in such a computer’ s interface, e. g., by adding speech input. There are no absolute criteria Which can be used to decide Whether a particular device is a device With a limited user-interface or not. There is alWays room for improvement and thus any computer device per se is assumed to use, ie “richer” (in that it has more features), more intui tive, faster, or the like. The user-interface description 15 is then processed by the second device 12 such that the more poWerful user-interface 19 is then displayed and operated via the second device 12. Then user-inputs and/or commands and/or parameters are sent back to the ?rst device for execu tion. In the present example, commands 17 (<command>) and parameters 18 (<parameters>) are sent back to control or operate the ?rst device 10 in response to commands input by a user on the second device 12.

The ?rst device 10 may provide its user-interface in some to be a device With a limited user-interface. The present inven tion may be implemented Where there is a second computer device that has a more poWerful user-interface, more adequate, more convenient, or superior user-interface capa bilities. Not all aspects of the user-interface have to be supe standard format (herein referred to as a user-interface descrip tion 15) to be broadcast to all other devices, such as the second

device 12, appearing in vicinity. If the user-interface descrip

20 tion 15 is suf?ciently small then the entire interface descrip tion can be transmitted quickly and stored at these other rior or more powerful. It is suf?cient, for example, if there is a ?rst device Which has no speech input (i.e. it has a limited user-interface) and a second device Which has a speech input.

The present invention may also be implemented, as selected

25 by a user, to control the second computer device that has a devices.

If there are a plurality of devices (“controllable” devices)

With limited user-interfaces in vicinity of a second device

With a superior user-interface, then the user requests some visual veri?cation on the second device listing all “control more poWerful user-interface via the device With the limited user-interface, if so desired as a users convenience dictates. lable” devices (e.g., in the form of a list, a menu, a graph, or the like) from Which he/ she can then choose one device With

Some of the above-mentioned devices can be regarded hereinafter as the device (controller) Whose interface is used

30 to interact With the user-interface limited device (controlled a limited user-interface and request its user-interface to be

displayed, thereby beginning the aforementioned process,

according to the present invention. device). The communication path 16 betWeen the ?rst device 10 and second device 12 is used to transfer data for a speci?c com device With a limited user-interface Where, for example, one or more of the folloWing applies: the user-interface is inad

35 mand from the ?rst device 10 (thus becoming the “controlled device” or server) to the second device 12 in vicinity (the controller or client/user agent). The second device 12 pro equate for the tasks required; the user-interface is small and dif?cult to read, understand, or hear; the user-interface pre sents an inconvenience to the user; there is no graphics capable display (e. g. a text-only display); there is a restricted number of input keys, or input keys Which are too small; there

40 vides the user-interface description 15 to the user. This can be done by displaying it (reference number 19) to the user on display 14, for example. Then, the second device 12 aWaits the user’s interaction.

The user enters responsive commands, eg by picking of buttons and thus imposes complicated control structures that make it dif?cult to operate the device Without prior exten sive study of a user’s manual, especially for seldom used or advanced functions; the user-interface is not poWerful

45 enough, making its use to sloW, or has loW resolution, or the like.

Devices With a more poWerful user-interface capabilities his/her choice from a presented menu, or supplies input by keying-in the requested data. In doing so the user makes use bilities (be it a larger keyboard 13, voice-recognition, color display 14, or the like). Information describing the user’s interaction, selection, or input is then sent back to the con trolled device 10 in the form of “requests” (i.e. commands 17 generally ful?ll, for example, one or more of the folloWing criteria: there is a larger screen; there is a screen With graphics and4optionallyione or more parameters 18) via the com

50 munication path 16. capability; there is a full keyboard; there is a pointing device; there is a voice-input feature and so forth.

The controlling device 12 is not required to have any prior knoWledge of the features and the user-interface of the con

The user-interface may be any kind of interface used for interaction betWeen a user and the device, such as a display,

keyboard, mouse, track point, audio output, speech recogni

tion input, tactile input, etc. trolled device 10. No special softWare needs to be pre-in stalled because everything is dynamically doWnloaded from

55 the controlled device 10 When required. Any Laptop or PDA that happens to be handy, or maybe even a public kiosk is illustrated in FIG. 1. There is a ?rst computer device 10

Which has a limited user-interface 11 (in the present example the user-interface comprises a simple display and a feW but tons). When there is a second computer device 12 in vicinity

60 of the ?rst device 10 that has superior user-interface capabili ties (keyboard 13 and display 14) than that of the ?rst device system, could quickly be used as a “user-interface server”

Without installing anything and Without leaving any notice able traces on that system (except maybe a feW modi?ed cache-entries in the system’ s memory). HoWever, there are of course some prerequisites Which must be ful?lled by all involved devices to alloW the aforementioned interfacing.

One such prerequisite requires that both devices have a

standardized procedure and format of describing suf?ciently

10 and Which may be controlled or con?gured, and assuming that the tWo devices 10 and 12 communicate With each other, the superior user-interface capabilities of the second device

65 rich user-interfaces such that it is possible to render typical

user-interface controls, i.e. display input-prompts, selection

12 may be employed to facilitate and speed-up the use of the menus, help-texts or other text-messages to visualiZe a

US 7,577,910 B1

9 10 device’s status, etc. Suitable candidates for such user-inter face description formats are: HTML (the HyperText Markup guage de?ned by the WAP forum), other, still-to-be-de?ned

XML (Extensible Markup Language) dialects, and X-Win doWs protocol.

One preferably uses a user-interface description Which is optimized so that transmissions betWeen devices are ef?cient.

The user-interface description is preferably ?exible and extensible.

The present invention implements any suitable design used for broadcasting and/or discovery of service offerings of the devices as described above, Without being limited to any one particular design. What is required is that a device having a more poWerful user-interface receive and store service offer ing information describing those devices in vicinity having a limited user-interface. The service information is updated frequently, since the Wireless netWorks composition may

change frequently.

In addition, the devices must be able to detect each other’ s

presence, exchange basic capability descriptions andion

devices Within vicinity to determine Whether certain services demandibe able to setup a suf?ciently reliable point-to point connection amongst each other. This basic capability device maintains a record With information about services and associated identi?ers about a second device, Which acts description can be a simple ?ag or bit combination, for example, Which describes standard types of services. These as a service-providing device. The ?rst device may comprise a service discovery module Which maintains a record With standard types of services may be prede?ned such that they can be identi?ed by a simple ?ag or bit combination. The basic capability description can also be any other kind of information about services and associated identi?ers, and a list of identi?ers about service-providing devices. The service discovery module enables the ?rst device to distinguish a information Which is Well suited to identify one or several of

20 the services offered. In addition to identifying a type of ser service offered by a service-providing device in vicinity from a service offered by a service-providing device not in vicinity. vice, one might have to set or de?ne certain parameters and The resource discovery design may be de?ned such that, options (referred to hereinafter as parameters).

When used in combination With a Wireless communications

Thus, there are certain common requirements betWeen protocol, it controls certain services or tasks carried out or devices. The devices, for example, Will initiate a resource

25 discovery design and exchange a capability and/or device class description for describing their respective service offer assigned to a device With a superior user-interface Which is in vicinity of the device With a limited a user-interface request ing the service. the ability to detect the presence of other potential devices in

vicinity by receiving communications from the other poten

30

The device’s capability description may include a basic initial message (a “pointer”) Which is stored at the receiver side as part of each device’s description, for example. tial devices, and upon detection of other devices in vicinity,

upload information describing their service offerings. In

addition, it is advantageous if a device is able to signal its broadcast of service information may optionally be done using a different protocol layer than the layer eventually used

The controlled device sends user-interface descriptions in some standardized format and receives and interprets inputs, commands and/or parameters sent back from the controlling device. presence to other devices and to doWnload its oWn service offering information or (broadcast it) to the other devices. The

35

The controlling device understands and is able to receive the user-interface descriptions and to make them accessible to the user on demand or automatically, eg by displaying a for sending user initiated command information to remotely control the controlled device 10. In another approach, the service information is inherently knoWn based on Which ser

40 menu or a list of all devices currently in vicinity. The control ling device then forWards commands to the controlled device.

An example of the present invention is illustrated in con nection With a preferred embodiment hereinbeloW. In the

folloWing description, the invention is implemented and

C and D, for example).

An example of a design for broadcast and/ or discovery of service offering information is addressed in a co-pending

45

European patent application entitled “Service Advertise

ments in Wireless Local Networks”, ?led on 25 Jan. 1999,

currently assigned to the assignee of the present application,

the contents of Which are hereby incorporated by reference. that Will soon be available on many mobile devices; WML’s

“deck-of-cards” metaphor maps perfectly to familiar con

?guration paradigms (“property-sheets” or “tabs”); WML

According to this design, each device takes turns broadcast ing a list of services (among Which could be the ability to send

50 tokeniZed WML. Most tags become single-byte items and strings are collected in a string-table. In the example given user-interface description and receive the corresponding commands) available. The general approach is that a group of devices Will take turns broadcasting a list of service offerings

(hereinafter referred to as user-interface description) avail able. By using variable transmission delays that are reset

55 form is only 652 bytes, even Without compressing the string table. If the string table Were compressed (e. g., using Lempel

Ziv, the same algorithm used in the popular .Zip or .gZip

When other simultaneous device broadcasts are received, and adjusting the distribution of these delays, neW devices in

vicinity are quickly identi?ed, and devices leaving vicinity

tools), the resulting ?les are even smaller. In addition, WML has built-in variables replacement functionality; and WML has timer functionality.

In order to signal that certain commands (and parameters) are quickly identi?ed as no longer in vicinity. This design

60 provides for the formation of small separate ad-hoc groupings of connected devices. Local netWorks are established imme are to be sent to the controlled device, the WML-broWser (or its underlying communication stack) must recogniZe URLs that use a special “protocol”.

diately (ad-hoc) When needed, and change as the grouping

varies according to the devices currently in vicinity. A net

The protocol may be based on existing protocol, like:

Work of all eligible devices in vicinity (devices that Will alloW

65

HTTP (HyperText Transport Protocol, i.e. the Intemet themselves to be netWorked) can be established While alloW ing devices to be added or removed accordingly.

[RFC2068], in example:

US 7,577,910 B1

11 12

FTP (?le transfer protocol) in example:

The gopher protocol, in Which there have already been suggestions to extend the notation using additional schemes

in example:

sony_cdpi990_mmoser_livingroom Which is a logical name that has to be mapped to the physical address of the device, and the remainder (command and optional parameters) Which is the actual request to be sent to the device. In the above example, the command “main_menu” is issued (Without parameters) to return to the initial main menu.

The “?le:”-protocol, Which has already been Widely

adopted and incorporated into most WWW-browsers in

example:

The above URL is stored as part of the description that all devices maintain about other devices currently in vicinity. If the user-interface description ?le siZe is small enough, the device may send the entire user-interface description imme

diately folloWing the resource discovery phase.

(Note: the //<host_name> fragment is optional andiif not presentidefaults to “this host” or “local host” Which

Devices With a means to display the user-interface descrip tion act as service providers and controllers for devices With accesses a local ?le rather than a ?le or resource located at a a limited user-interface. If there are a plurality of devices With remote server).

Similarly suggested, but not yet adopted, is a protocol that alloWs the control of and communication via “local” devices

(serial ports, printer ports, smart-card readers, USB-ports, etc.) using for example:

controller, the plurality of devices together With their user interface capability-description may be displayed on the con troller’s screen for selection. The controller’s display may

20 typically include a system-menu With a selection button labeled “act as user-interface for a nearby device”. Clicking the selection button pops up a list of “controllable” devices. e.g. device://COM1/setbaudrate;19200 to change the ?rst communication port’s serial speed to 19200 baud/ s.

The user then picks a controllable device from that list,

Whereupon the user-interface-URL (Wml-user-interface

In addition, the present invention proposes a <comm> pro tocol to send commands to communicating devices that are

“attached” using some short-range communication means

25

URL) is transmitted to the selected device to carry out the method according to the present invention.

The transmission of the user-interface-URL triggers the transfer of the main-control menu of the controlled device

(like IR or RF) using the folloWing protocol: <comm>://

<device_id> [: <portnr>]/<pre?x>/<cmd>[ ;<params>.]

from the controlled device to the controlling device (control

30

Which this command/request is to be sent, e.g. “IrD ” or is needed to provide communication means, Which support multi-par‘ty communication (i.e. not only 1 :1 communication

35

“property sheets”, a visualiZation technique that is often used to edit object attributes and parameters.

An example is illustrated in FIGS. 3-5, Where a CD-player is controlled by a controller having a display 31. The user interface 30 shoWn contains a deck With only 4 cards: a eter <portnr> may also be added to specify a speci?c port,

Where more than one communication channel exists betWeen card to select the play mode 34 and a generic help card 37. The communicating devices, or to select a non-default channel for special purposes (eg for device monitoring, diagnosis, con

?guration, etc.).

40

When a WML-browser is used by the controller, the deck layout is displayed as WindoW 30 on the screen 31 of the

This device-id may be derived from the concatenation of a controller, as shoWn in FIG. 3. When, for example, the user clicks on the CD-labels link 32 or the user selects the “CD manufacturer identi?cation and a model identi?cation (e.g.

“sony_cdpi990X”) With some user-speci?ed arbitrary name or physical location (eg “mmoser_livingroom'”)

45 names”-tab 33, a card 40 to edit CD-titles is displayed, as shoWn in FIG. 4. Here the user may enter a CD-name in an insert ?eld 41 using the input facility of the controller device, a URL-<path>) is used to group commands into some tree structure, e. g. if one looks at a printer’s menu-tree (execution e.g., a full-bloWn keyboard, pen-input, voice-input, or the like.

Selecting the play-mode link 35 (or play_mode-tab 34)

tree) as shoWn in FIG. 2.

A parameter <cmd> (or the last path-fragment) speci?es

50 the actual command, While a parameter <params> describes

optional parameters of the above command. Examples imple

pops up a WindoW 50 as shoWn in FIG. 5. Again, the user radio buttons 51 using the controller device’s input facility.

The commands and parameters are transmitted on user menting this protocol are represented beloW: bluetooth://so

ny_cdpi990_mmoser_livingroom/cd_titles/add;BMW:

Exodus bluetooth://sony_cdpi990_mmoser_livingroom/ play_mode/ select; shuf?e.

55

To perform resource discovery, a communicating device sends a command string in the folloWing format:

Wml_ui:<device_id>/ [<path>]/<command>[ ;<param

command, e.g., the user clicks the OK-button 52 on the CD player play-modes WindoW 50 of FIG. 5, instructing the con troller’s broWser to submit the folloWing URL, in example:

bluetooth://sony_cdpi990/playmode/select;Normal

The above URL Would sWitch the play mode of the CD player to “Normal”. Based on the protocol (here “bluetooth”) the controller device’s communication stack recogniZes that

eters>]

An example implementing this protocol is shoWn beloW:

Wml_ui:sony_cdpi990_mmoser_livingroom/main_

menu

60 this is not a normal request, to be sent out via TCP/IP and the

Internet, but rather that this request must be intercepted and forWarded to the local (bluetooth) communication stack.

The host speci?cation of the URL is then used to address

Resource discovery is performed so a device may describe its resources to its peers. In the above example Wml_ui is a

65 prede?ned service name (standardized) and sony_cdpi

990_mmoser_livingroom/main_menu the speci?ed device (here a “Sony CD-player model 990”) and the remaining URL-part (the optional path, the command

and the optional parameter(s)) are then sent to the speci?ed device.

US 7,577,9l0 B1

13

14

The addressed device must have a simple “command-in terpreter” that is able to recognize and execute received com mands by analyzing submitted URLs, i.e. extracts and recog

niZes certain command strings plus optionally separates and

converts parameters, etc. The complexity and robustness of

Ware interface 25. The device 70 may also have a user-inter face unit 24 for interaction With a user (eg a small LCD display and some input keys).

During remote access, user-interface information is sup this interface is completely up to the manufacturers discre tion, and therefore may vary according to manufacturer and model number.

The user may optionally receive a visual or audible con?r and transmitted to the (remote) controlling device via trans mitter driver 73 and channel 81. User-interface information comprises any information needed by the controlling device

With the more poWerful user-interface capabilities in order to provide a superior user-interface to the controlled device for thus “submits a request” to the controlled device. For this reason the controlled device may react (this is an optional a user. Depending on the implementation, the user-interface information may be information that describes a full user step) and return some response to the submitted request, since simply receiving a time-out message in the broWser With no interface (see item 19 in FIG. 1), or describes a partial user interface (item 63 in FIG. 6).

Commands entered at the controlling device by the user are success/failure indication Whatsoever is generally not su?i cient.

The ?exibility, siZe, and complexity of this response is received via channel 82 by the receiver driver 74 and for

Warded to MAC 72 and to user-interface manager 71. To completely up to the manufacturers discretion, the devices capabilities, and resources. For example, the device may: control the device 70 the user-interface manager 71 may communicate directly (item 83) or indirectly via an optional return data to display a speci?c card con?rming the reception

20 of the command and describing the results of its execution; return data to display the complete user-interface-“deck” again With certain texts or default choices noW adapted application programming interface 79 (API) and a device speci?c application 78 With hardWare drivers 26, and conse quently With device speci?c hardWare 20 that provides and embodies a device’s purpose and/or functionality (be it a according to the status changes caused by the previous com mand; or return just a minimal OK, or error, page depending

25 on the command’s result. The user may then navigate back to the control stack by pressing “return” in the broWser. Many video cassette recorder, a coffee machine, a printer, a stereo device, etc.). The actual activity or functionality of the device

70 is independent of the present design; using the present design provides that this activity may be controlled and/or monitored from another device. other responses or combinations of the above responses are

possible.

One exemplary implementation of the present invention is

30 application 78 are logical constructs. They may be imple described in connection With FIG. 7A. FIG. 7A is a block

diagram illustrating the components, both logical compo

nents and physical components, of a device 70 With a limited mented on separate devices or incorporated into a program stored in memory 76. If incorporated into a program, the device 70 may physically be the same as other devices, except

Ware layer. The device 70 comprises a transmitter driver 73

35 for sending information via an output channel 81 to another for the fact that it comprises the above-mentioned program.

The program comprises instructions that When processed by the CPU 77, instruct the device 70 to perform the operations device, such as a device With more poWerful user-interface capabilities, and a receiver driver 74 for receiving information from other devices via input channel 82. The channels 81, 82 necessary to implement the present invention.

The user-interface manager 71 exchanges the user-inter face information, alloWing the user-interface to be provided may be any kind of channels, such as an IR, RF, or body

40 netWork channels, for example. These channels do not have to be the same. It is conceivable, for example, that the output channel 81 is an infrared channel Whereas the input channel to the remote controlling device and control information and/ or parameters to be received from the device in response to user input.

82 is a RF channel.

The transmitter driver 73 and receiver driver 74 communi

45 steps that are performed by a device 70 With a limited user interface. The device 70 also transmits service information cate With a medium access control (MAC) unit 72. The MAC layer is Well de?ned by international standards (cf. ISO OSI

(service announcements) to one or more devices Within vicin ity periodically in step 84. This service announcement pro

(Open Standards Interconnection) reference model as described in A. S. Tannenbaum’s book “Computer Net ventional unit employed in communication systems to con trol the MAC layer. Note that a MAC layer is a logical division, and Would be only logically divided from other parts of the protocol implemented at 71 on the same physical cess may optionally run in the background, as indicated by the dashed loop 8411. When the device receives information from

50 another device (second device), it categoriZes the information into one of several categories. In the present example there are three categories: service information; request to send user interface (UI) information; and user input. When the input contains service information, then the received information is avoid collisions betWeen transmitted data packets received simultaneously from different devices. In the present embodi

55 used to update the device’ s oWn list of services in step 89. The service information may be used by the second device to transmit information about its capabilities. The service infor mation is stored in a list for later retrieval. Alternatively, the information may be retrieved only if needed, for example.

Device 70 and it’s components may be poWered via a

60 poWer plug, a solar cell, a battery, or any other suitable means

Which are purposely left out for clarity.

With reference to FIG. 7D, the device 70 may comprise a

When the input is identi?ed to be a request to send user interface (UI) information then the device sends its UI infor mation to the second device in step 85. Finally, When the information comprises a user input received from the second bus 21 enabling communication betWeen some of the device’s components/units, such as the central processing

65 unit (CPU) 77, memory 76, the communication hardWare 22,

23 and any other device speci?c hardWare 20 through a hard device, then the device 70 processes and/or executes this user input in step 86. In an optional step 90, a con?rmation signal is returned to the second device to indicate to the user that the controlled device processed or executed the command. Alter

US 7,577,910 B1

15 16 natively (see the dashed arrow at the lower right-hand side) mation is not received, the controller device 700 returns to the device 70 may, if instructed, send an entire or partial UI background monitoring 102 to receive service information in description again Which is updated to re?ect results or status step 103. changes caused by the prior command. Finally, the device 70

In an alternative embodiment, the controlled device ini returns to monitoring mode in step 87, and aWaits additional tiates the process. In this case, the controlled device sends information from the second device or any other device in

vicinity.

user-interface information to a particular second device. If there is a plurality of devices Within vicinity of the controlled

The embodiment described in connection With FIG. 8 device, then the controlled device or the user selects one. implements a procedure Where the second device (i.e., the

Before user-interface information is sent out, the device or device With a superior or more poWerful user-interface) trig gers the device 70 With a limited user-interface to send user user may determine Whether there is another device in vicin ity Which has the best user-interface. This may be done by interface information, Which may, for example, be initiated

When a user points the second device into the direction of the computer device 70 With limited user-interface, or simply brings the second device in vicinity of the device 70. simply looking at the information stored in a list of available devices in vicinity With corresponding services. If no list is maintained, the controlled device transmits the user-interface information hoping there is a device in vicinity Which is able to receive and interpret the user-interface information. The second device receives the user-interface information and a controller device 700ii.e. a device that has a more poWer ful user-interfaceiis illustrated. FIG. 7B illustrates typical logical and softWare-layer building blocks and FIG. 7C illus provides a corresponding user-interface to the user. The user then uses this user-interface to input information. The user’s trates building blocks of the hardWare layer. As shoWn in FIG.

73, the controller device 700 comprises a MAC protocol

20 input is then sent to the controlled device Where the input is feedback is returned to the second device to indicate to the handler 720, a transmitter driver 730, and a receiver driver

740 for communication With a remote device (not shoWn). In addition, there is a user-interface manager 710 and a user interface driver 750 to communicate With the user-interface.

25 processed and/or executed. In an optional step, con?rmation user that the controlled device processed or executed the command.

An extension of the above described design is described beloW With reference to FIG. 6. The user-interface source 67

As illustrated in FIG. 7C, the device 700 comprises a bus

706 (e. g. a back plane bus or a cable bus) for interconnecting

6. The controlled device 60 does not necessarily have to a transmitter 701, receiver 702, memory 703, CPU 704, and user-interface 705 connected to a display and/or keyboard or,

supply the entire user-interface description (Which could

30 become quite large4e.g. When lots of graphical elements are

pointing device, for example.

employed) but only minimal required information. In this

FIG. 9 is a flow chart illustrating the corresponding steps

performed by the controller device 700 during operation. The

case, the device 60 just sends a partial user-interface 63 (eg a text-only version) via the Wireless communications path 66 controller device 700 background monitors 102 and receives service information from a device With a limited user-inter

35 face in step 103. If there is a plurality of computer devices

With limited user-interfaces (controllable devices) Within vicinity of the controller device 700, then a list of these to the controller 62, or the controlled device 60 supplies only an initial user-interface description 63 (eg a URL) or

“pointer”.

The actual user-interface 65 or the additional information

(eg graphics 69) are then retrieved from other locations, for controllable devices is displayed for the user in step 92. In step 91 the user may also optionally request the list. The user

40 example, a ?le pre-installed on the controller or a WWW server on the Internet, and combined into a uni?ed user then selects a controllable device he Wants to control (con trolled device) or interact With from the list in step 93. If there is only one controllable device, or if the controller device 700 otherWise knoWs Which controllable device the user Wants to interact With, then steps 91-93 may be bypassed as indicated

45 by arroW 94. Next, a request to send user-interface informa tion is transmitted to the controlled device in step 95. The interface presentation 68 on the screen 14 of the device 62. In the illustrated example the actual user-interface is fetched alloWs the user-interface description 63 stored in the device

60 to be very small and only requires the implementation of a

simple command and parameter parsing capability in the

device 60 itself. controlled device responds by sending the requested user interface information, as illustrated in step 85 of FIG. 8. After

Another, more complex user-interface implementation alloWs the controlled device to implement only basic com the user-interface information is received by the controller

50 device in step 96, a corresponding user-interface is displayed mands, using scripting techniques. These commands are combined by the controller to form more poWerful compound to the user in step 97. This may, for example, be done by displaying the user-interface to the user, by reading text to the commands and also automate certain repetitive tasks. A broWser equipped With some ?exible scripting language user, by printing some information, etc. The controller device

700 then Waits to receive user input in step 98, and corre

55 sponding commands are then sent back to the controlled

Would, for example, automate tasks like the folloWing:

Ask the user to insert a CD device in step 99. A con?rmation signal may optionally be received from the controlled device and provided to the user identi?cation code

Search that CD’s ID-code in a Web-based database (eg

(not illustrated in FIG. 9). The controller device 700 then

Waits for additional user input for a predetermined period of

60 time in step 100, returning to step 98 if additional user input is received. If no additional user input is received during the predetermined period in step 100, the controller device 700 then aWaits additional user-interface information in step 101 for a predetermined period of time. If the additional user

65 interface information is transmitted the interface information is received, returning to step 96. If the user-interface infor

“CDDB”)

CD-player’s User Interface and submit that request.

Start over.

CD-player (controlled device in the present example) While

the PC (controller device in the present example) searches and automatically programs the corresponding CD-labels into the CD-player.

US 7,577,910 B1

17 18

The present invention may also incorporate the translation pointing device provided by the controller device to control of user-interface controls. In this case, the controlling device features of a device Which lacks such a mouse, pen, or other acts as a format translator, i.e. converts user-interface ele

pointing device.

ments to/from different formats or media. The controlling

The present invention may in part or as a Whole be imple

device may, for example, provide speech synthesis and “read”

some text message to a vision impaired or occupied person

(e.g. during car-driving). Similarly, spoken commands to

mented by or on a special computer device or a general purpose computer device. This may be done by implementing the invention in some form of computer program. Computer enter data into an input ?eld or activate a control-element (a program in the present context means an expression, in any spoken “button click”) may be converted. Such conversion language, code or notation, of a set of instructions intended to involves user-interface representation formats that do not make any assumptions about the actual physical user-inter cause a computer device to perform a particular function either directly or after either or both of the folloWing: face capabilities available in a device, but rather specify a) conversion to another language, code or notation; and b) reproduction in a different material form. abstract functional levels of controls, e.g. WML does not specify minimal display siZes in pixels nor require a mini

While the present invention has been described in detail mum number of fonts to be available for menus and text

output, but rather speci?es “selection”, “input” and “activa

tion” capabilities. Menu-texts may eitherbe displayed or read

With reference to the preferred embodiments, they represent more exemplary applications. Thus, it is to be clearly under stood that many variations can be made by anyone having ordinary skill in the art While staying Within the scope and to the user and the user can type his/her ansWer or simply speak to the device. spirit of the present invention as de?ned by the appended claims. Although speci?c terms are used herein, the descrip

If a manufacturer relies on the fact that seldom used func tions need not be controlled via a device’s front-panel but

20 tion uses terminology in a generic and descriptive sense only and not for purposes of limitation. rather using a better suited external device, the amount of user-interface code for complicated, seldom used functions

The invention claimed is: can be drastically reduced, Which yields much easier, less error-prone softWare development and consequently shorter time-to -market and considerable price bene?ts due to quicker development. A controllable device in accordance With the present invention is simpler and less expensive.

25

Given a minimal communication range (say 15-20 m), the device to be controlled need not even be in the same room. It could be in the basement or on the roof (e.g. heating, air

30

condition, antennas, cable tuners, satellite receivers, etc.).

a limited user-interface by using at least one second device,

Wherein the ?rst and second devices communicate via a Wire nications protocol, the method comprising the steps of: transmitting the limited user-interface information from the at least one ?rst device to the at least one second

device;

providing an extended user-interface on the at least one second device, the extended user interface having more

The standardized communication channel betWeen a con trolled device and a controller can be extended (this technique extensive capabilities than the capabilities of the limited user-interface of the at least one ?rst device the extended is usually knoWn as “proxy”) to alloW the bridging of larger distances and to alloW remote-control and remote-diagnosis

capabilities. For example, building and heating control might

require special knoWledge such that even a good user-inter

35 user-interface utiliZing the transmitted limited user-in terface information and comprising extended functions so as to extend the capabilities of the limited user-inter face does not enable a customer to correctly adjust certain settings. By temporarily relaying the user-interface to a spe

face;

accepting user commands input via the extended user cialiZed ?rm, some external specialist may con?gure or diag nose a remote system. Such World-Wide access to home

40

interface;

transmitting user commands from the second to the ?rst equipment is convenient for a “normal” user, too, because it

Will alloW the user to control home systems from a remote location. alloWs more ?exible, more poWerful user-interfaces but

45 device; and executing the transmitted user commands on the ?rst device.

2. The method recited in claim 1, Wherein the user-inter face information is a standardized user-interface description.

3. The method recited in claim 1, Wherein the second tation. Any other markup language may also be used. device transmits a list of available services to the ?rst device

If the host-device (the controller) supports drag-and-drop

prior to said ?rst device transmitting user-interface informa capabilities, this functionality may be exploitedieg. to copy an appointment from a PDA’s agenda to the doWnloaded

50 tion to said second device. user-interface of a Wrist-Watch. The drag and drop capabili

4. The method recited in claim 1, Wherein the Wireless communication channel is automatically established betWeen ties Work betWeen host-applications and doWnloaded user the ?rst device and the second device. interfaces, as Well as betWeen devices. If more than one

5. The method recited in claim 1, Wherein the second device is currently controlled by the same controller, that host may act as an intermediate, i.e. one could drag and drop

55 device comprises a display for displaying said extended user interface. information betWeen tWo controlled devices (eg copy a phone number stored in a Wrist-Watch to a mobile phone).

6. The method recited in claim 1, Wherein the second device comprises a keyboard for accepting the user com

Many of the bene?ts of the present invention become obvi mands. ous When reading the speci?cation. The present design alloWs

7. The method recited in claim 1, Wherein a markup lan a user to make use of larger, better readable displays (e.g., a

larger color graphics display), superior/faster input capabili

ties (e.g., a full-?edged keyboard, or pointing device), and guage is used for user-interface information.

8. The method recited in claim 7, Wherein Wireless Markup better suited l/O interfaces (eg a printer or audio system). It

Language (WML) is used as the markup language. is certainly easier to program a mobile phone or a Wrist-Watch

65

9. The method recited in claim 1, Wherein the second device provides the extended user-interface by using broWser ing-in data on a tiny numeric keyboard. According to the softWare to display at least a portion of the user-interface information.

Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertisement