WO2012089360A1 - Electronic synapses for reinforcement learning - Google Patents
Electronic synapses for reinforcement learning Download PDFInfo
- Publication number
- WO2012089360A1 WO2012089360A1 PCT/EP2011/068183 EP2011068183W WO2012089360A1 WO 2012089360 A1 WO2012089360 A1 WO 2012089360A1 EP 2011068183 W EP2011068183 W EP 2011068183W WO 2012089360 A1 WO2012089360 A1 WO 2012089360A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- electronic
- electronic synapse
- synapse
- synaptic
- neuron
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/06—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N3/063—Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means
- G06N3/065—Analogue means
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11C—STATIC STORES
- G11C11/00—Digital stores characterised by the use of particular electric or magnetic storage elements; Storage elements therefor
- G11C11/54—Digital stores characterised by the use of particular electric or magnetic storage elements; Storage elements therefor using elements simulating biological cells, e.g. neuron
Definitions
- the present invention relates to neuromorphic and synapatronic systems, and in particular, tp am electronic synapse and a synapse cross-bar array.
- Neuromorphic and synapatronic systems are computational systems that permit electronic systems to essentially function in a manner analogous to that of biological brains.
- Neuromorphic and synapatronic systems do not generally utilize the traditional digital model of manipulating 0s and Is. Instead, neuromorphic and synapatronic systems create connections between processing elements that are roughly functionally equivalent to neurons of a biological brain.
- Neuromorphic and synapatronic systems may be comprised of various electronic circuits that are modeled on biological neurons.
- the point of contact between an axon of a neuron and a dendrite on another neuron is called a synapse, and with respect to the synapse, the two neurons are respectively called pre-synaptic and post-synaptic.
- the essence of our individual experiences is stored in conductance of the synapses.
- the synaptic conductance changes with time as a function of the relative spike times of pre-synaptic and post-synaptic neurons, as per spike-timing dependent plasticity (STDP).
- the STDP rule increases the conductance of a synapse if its post-synaptic neuron fires after its pre-synaptic neuron fires, and decreases the conductance of a synapse if the order of the two firings is reversed.
- an electronic synapse is configured for interconnecting a presynaptic electronic neuron and a post-synaptic electronic neuron.
- the electronic synapse comprises memory elements configured for storing a state of the electronic synapse and storing meta information for updating the state of the electronic synapse.
- the electronic synapse further comprises an update module configured for updating the state of the electronic synapse based on the meta information in response to an update signal for reinforcement learning.
- the update module is configured for updating the state of the electronic synapse based on the meta information, in response to a delayed update signal for reinforcement learning based on a learning rule.
- the invention provides a system, comprising a plurality of electronic neurons and a cross-bar array configured to interconnect the plurality of electronic neurons.
- the cross-bar array comprises a plurality of axons and a plurality of dendrites such that the axons and dendrites are transverse to one another.
- the cross-bar array further comprises multiple electronic synapses, wherein each electronic synapse is at a cross-point junction of the cross-bar array coupled between a dendrite and an axon, each electronic synapse configured for interconnecting a pre-synaptic electronic neuron and a post-synaptic electronic neuron.
- a preferred embodiment of the invention provides a space-division multiple access electronic synapse comprising a 6-terminal device with two terminals for reading, two terminals for setting and two terminals for resetting.
- FIG. 1 A shows a diagram of a neuromorphic and synapatronic system having a cross-bar array of electronic synapses, in accordance with an embodiment of the invention
- FIG. IB shows a diagram of an electronic synapse at the cross-point junction of a presynaptic path and a post-synaptic path, in accordance with an embodiment of the invention
- FIG. 2 shows a diagram of an electronic synapse at a cross-point junction involved in a read operation, in accordance with an embodiment of the invention
- FIG. 3 shows a diagram of an electronic synapse at a cross-point junction involved in a STDP-set operation, in accordance with an embodiment of the invention
- FIG. 4 shows a diagram of an electronic synapse at a cross-point junction involved in a STDP-reset operation, in accordance with an embodiment of the invention
- FIG. 5 shows a diagram of an electronic synapse at a cross-point junction involved in a STDP-set operation, in accordance with an embodiment of the invention
- FIG. 6 shows a diagram of an electronic synapse including an array of junctions, in accordance with an embodiment of the invention
- FIG. 7 shows a diagram of an electronic synapse involved in a STDP operation for an R bit, in accordance with an embodiment of the invention
- FIG. 8 shows a diagram of an electronic synapse involved in a STDP operation for a G bit, in accordance with an embodiment of the invention
- FIG. 9 shows a diagram of an electronic synapse involved in a STDP operation for a B bit, in accordance with an embodiment of the invention.
- FIG. 10 shows a diagram of a cross-bar array of electronic synapses, in accordance with an embodiment of the invention.
- FIG. 11 shows a diagram of an electronic synapse, in accordance with an embodiment of the invention.
- FIG. 12 shows a diagram of a static random access memory (SRAM)-based electronic synapse, in accordance with an embodiment of the invention
- FIG. 13 shows a diagram of a dynamic random access memory (DRAM)-based electronic synapse, in accordance with an embodiment of the invention
- FIG. 14 shows a high level block diagram of an information processing system useful for implementing one embodiment of the present invention.
- Embodiments of the invention provide electronics synapses configured for reinforcement learning (RL). Embodiments of the invention further provide neuromorphic and
- synapatronic systems including cross-bar arrays which implement spike-timing dependent plasticity (STDP), utilizing such electronics synapses for RL.
- STDP spike-timing dependent plasticity
- FIG. 1 A there is shown a diagram of a neuromorphic and synapatronic system 10 having a cross-bar array in accordance with an embodiment of the invention.
- the cross-bar array may comprise an "ultra-dense cross-bar array” that may have a pitch in the range of about 0.1 nm to 10 ⁇ .
- the neuromorphic and synapatronic system 10 includes a cross-bar array 12 having a plurality of neurons 14, 16, 18 and 20. These neurons are also referred to herein as "electronic neurons”. Neurons 14 and 16 are axonal neurons and neurons 18 and 20 are dendritic neurons.
- Axonal neurons 14 and 16 are shown with outputs 22 and 24 connected to axon paths (axons) 26 and 28, respectively.
- Dendritic neurons 18 and 20 are shown with inputs 30 and 32 connected to dendrite paths (dendrites) 34 and 36, respectively.
- Axonal neurons 14 and 16 also contain inputs and receive signals along dendrites, however, these inputs and dendrites are not shown for simplicity of illustration.
- the axonal neurons 14 and 16 will function as dendritic neurons when receiving inputs along dendritic connections.
- the dendritic neurons 18 and 20 will function as axonal neurons when sending signals out along their axonal connections. When any of the neurons 14, 16, 18 and 20 fire, they will send a pulse out to their axonal and to their dendritic connections.
- Each connection between axons 26, 28 and dendrites 34, 36 are made through a synapse device 31.
- the junctions where the synapse device are located may be referred to herein as "cross-point junctions”.
- Neurons 14, 16, 18 and 20 each include a pair of RC circuits 48.
- axonal neurons 14 and 16 will "fire” (transmit a pulse) when the inputs they receive from dendritic input connections (not shown) exceed a threshold.
- axonal neurons 14 and 16 fire they maintain an A-STDP variable that decays with a relatively long, predetermined, time constant determined by the values of the resistor and capacitor in one of its RC circuits 48.
- this time constant may be 50 ms.
- the A-STDP variable may be sampled by determining the voltage across the capacitor using a current mirror, or equivalent circuit. This variable is used to achieve axonal STDP, by encoding the time since the last firing of the associated neuron, as discussed in more detail below. Axonal STDP is used to control "potentiation", which in this context is defined as increasing synaptic conductance.
- this time constant may be 50 ms.
- this variable may decay as a function of time according to other functions besides an exponential curve.
- the variable may decay according to linear, polynomial, or quadratic functions.
- the variable may increase instead of decreasing over time.
- this variable may be used to achieve dendritic STDP, by encoding the time since the last firing of the associated neuron, as discussed in more detail below.
- Dendritic STDP is used to control "depression", which in this context is defined as decreasing synaptic conductance.
- FIG. IB shows a perspective view of an electronic synapse 31 at the cross-point junction of a pre-synaptic path 26 and postsynaptic path 36, according to an embodiment of the invention.
- the two neurons are respectively called pre-synaptic and post-synaptic.
- a "read" signal is sent from the pre-synaptic neuron 14 to the post-synaptic neuron 20.
- the synapse 31 is STDP-set.
- the post-synaptic neuron 20 fires and then the pre-synaptic neuron 14 fires, the synapse 31 is STDP-reset.
- Reinforcement learning generally comprises learning based on consequences of actions, wherein an RL module selects actions based on past events.
- a reinforcement signal e.g., a reward signal
- the RL module then learns to select actions that increase the rewards over time.
- the STDP-set and STDP-reset operations do not take place immediately. Rather, if a reward ("value") signal occurs within a time window, then STDP-set or STDP-reset operations are applied.
- the synapse 31 implements multiple information bits.
- the synapse 31 maintains three bits including a bit R, a bit G and a bit B.
- Bit R is for read
- bit G is for STDP-set
- bit B is for STDP-reset.
- bits G and B are set to 0 as their natural state. If the presynaptic neuron fires and then the post-synaptic neuron fires, then for STDP-set the bit G is set (e.g., set to 1). If post-synaptic neuron fires and then the pre-synaptic neuron fires, then for STDP-set the bit B is set (e.g., set to 1).
- the post-synaptic neuron fires and then the post-synaptic neuron fires, then STDP-reset is applied to bits B and G.
- bits B and G are reset to 0 based on a time constant decay (e.g., 1 second).
- resetting bits B and G comprises a random process resetting B and G, independent of neuron firing.
- bit R is set and reset when a reward occurs as follows:
- a synapse 31 comprises an n x n array of junctions.
- Logic for reading bit R is at the periphery of the synapse 31 as shown by example in FIG. 7, further illustrating reading bit R of the synapse 31, wherein when a pre-synaptic neuron fires, it sends a read pulse to the post-synaptic neuron. Then the post-synaptic neuron asynchronously reads the pulses as they arrive from the pre-synaptic neuron via R junction of the synapse 31.
- Logic for set/reset of bit R is contained within the synapse 31. In one implementation, bit R may be implemented using DRAM devices.
- Logic for setting bit G is at the periphery of the synapse 31 as shown by example in FIG. 8, further illustrating setting bit G of the synapse 31, wherein when a post-synaptic neuron fires, it sends an alert pulse to the pre-synaptic neuron.
- the pre-synaptic neuron probabilistically sets a pre-synaptic set pulse.
- the post-synaptic neuron always sends a post-synaptic set pulse. If both pre-synaptic set and post-synaptic set pulses arrive at the junction for bit G together, then bit G is set.
- bit G has a preferred set value of zero, and resets after a certain time constant (for example, 1 sec).
- re-setting G comprises a random stochastic process the resets bit G, in a fully asynchronous fashion, independent of firing of neurons.
- the process has a mean resetting time of about 1 second and has a heavy tail distribution.
- the reset of G is initiated by pre-synaptic neuron.
- bit G may be implemented using DRAM devices.
- bit B is set. If the pre-synaptic neuron fires and then the post-synaptic neuron fires, bit G is set. The pre-synaptic neuron, when it fires, alerts a post-synaptic neuron. Depending upon when it last fired, the post-synaptic neuron probabilistically sets a post-synaptic set pulse. Presynaptic neuron always sends a pre-synaptic set pulse.
- bit B is set. Further, logic for re-setting B resides in the block 31.
- the bit B has a preferred set of zero and it simply resets after a certain time constant (e.g., about 1 sec).
- a random stochastic process resets bit B, in a fully asynchronous fashion, and independent of firing of neurons.
- the process has a mean resetting time of about 1 second and has a heavy tail distribution.
- the reset is initiated by post-synaptic neuron.
- bit B may be implemented using DRAM devices.
- the present invention provides a system 70 for implementing electronic reinforcement learning synapses according to an embodiment of the invention.
- the system 70 comprises an N x N cross-bar array of RGB synapse blocks 31 asynchronously operable in parallel (N rows and N columns).
- the system 70 further comprises N pre-synaptic neurons (e.g., Prel, Pre 2, Pre N) and N post-synaptic neurons (e.g., Postl, Post 2, Post N), interconnected via the cross-bar array of synapses 31.
- each post-synaptic neuron 31 comprises an electronic mixed- mode (analog-digital) asynchronous neuron.
- each electronic synapse 31 is configured for interconnecting a pre-synaptic electronic neuron and a post-synaptic electronic neuron.
- the electronic synapse 31 comprises memory elements (e.g., memory devices 31R, 31G, 3 IB in FIG. 11) configured for storing a state of the electronic synapse and storing meta information for updating the state of the electronic synapse.
- Each electronic synapse cell 31 further comprises an update module (e.g., module 31L in FIG. 11) configured for updating the state of the electronic synapse based on the meta information in response to an update signal for reinforcement learning.
- the update module is configured for updating the state of the electronic synapse based on the meta information, in response to a delayed update signal for reinforcement learning based on a learning rule.
- FIG. 11 illustrates an example implementation of an R, G, B synapse array 31 as a synapse cell (block) which can be operated in parallel with other synapse cells 31, without requiring phases (and without requiring time-division multiple access for read, set, and reset).
- Each synapse cell 31 can be operated completely asynchronously of other synapse cells 31, thus eliminating the need for a clock.
- each RGB synapse cell 31 comprises a digital complementary metal-oxide-semiconductor (CMOS) update logic 31L at the local synapse cell level can be used to write the R cell.
- the cell 31 comprises memory elements 31R, 3 IB, 31G for bits R, B and G, respectively.
- the memory elements can comprise static random access memory (SRAM), dynamic random access memory DRAM, Phase-change memory (PCM), magnetic tunnel junction (MTJ), etc.
- the synapse cell 31 comprises a space-division multiple access electronic synapse wherein the electronic synapse is represented as a 6-terminal device with two terminals for reading, two terminals for setting and two terminals for resetting.
- the update module 31L comprises a software module including computer readable program code to execute on a processor (e.g., information processing system 100 in FIG. 14), wherein the software module includes computer readable program code configured to update the state of the electronic synapse as described herein according to the embodiments of the invention.
- a processor e.g., information processing system 100 in FIG. 14
- the software module includes computer readable program code configured to update the state of the electronic synapse as described herein according to the embodiments of the invention.
- the R memory cell maintains the state of the synapse.
- the G and B memory cells maintain meta information used for a subsequent update of the synapse.
- the neurons determine the read/write information for the memory cells.
- the synapse 31 provides a connection from a pre-synaptic neuron to a post-synaptic neuron which collaborate to activate the appropriate word line and bit lines to read/write the R, G and B memory cells. Neurons only write the G and B memory cells externally using Write ports.
- the G and B memory cells are read internally by the update logic 31L using Read ports to accordingly update the R memory cell.
- the R memory cell is read externally using a Read port and written (i.e., updated) internally by the update logic 31L using a Write port.
- the state of the synapse can have one or more bits storing multiple values indicating level of conductivity of the synapse.
- R memory cell stores state of the synapse, wherein the state of the synapse is a 1-bit synapse (0 for a conducting state indicating a connection, 1 for non-conducting state indicating no connection).
- a neuron can determine a connection through a synapse by reading the R memory cell.
- the pre-synaptic neuron and the post-synaptic neuron coupled to the synapse implement a process to write the B and G memory cells for reinforcement learning.
- the neurons store update values into B and/or G memory cells using Write ports.
- an update value from the B or G memory cell is used to update the R memory cell as state of the synapse in response to an incoming reward signal.
- the R memory cell is updated with the value of the B memory cell or the value of the G memory cell depending on a later incoming reward signal as a reinforcement signal (delayed update), as described above.
- the STDP value is stored in a G or B memory cell, and at a later time the state of the synapse is updated by updating the R memory cell with the values from G or B cells.
- parallel word lines (horizontal) and bits lines (vertical) are used to access the memory cells.
- Each memory cell has a read word line, read bit line, a write word line and write bit line.
- the update logic 31L implements a logical exclusive or (XOR) combination of the B and G memory cell meta information, to update the R memory cell state of the synapse.
- the synapse cell 31 provides reinforcement learning with SRAM and DRAM implementation. Referring to FIG. 12, in an SRAM-based RGB cell implementation, each SRAM cell 31 is transposable (can be accessed by peripheral circuits in either rows or columns). Referring to FIG. 13, in a SRAM and DRAM-based
- data in DRAM memory elements decays over time to a base state.
- a clocking signal is used to clock the operation of the memory cells in the cross-bar array.
- the memory cells can be accessed synchronously or asynchronously.
- the synapse When the pre-synaptic neuron fires and then the post-synaptic neuron fires, the synapse is set. When the post-synaptic neuron fires and the pre-synaptic neuron fires, the synapse is reset, and if a reward (value) signal occurs within a time window, STDP-Set or Reset is applied.
- Electronic reinforcement of learning synapses further comprises: reading R rows in parallel, reading and setting G columns in parallel, resetting G rows in parallel, reading and setting B rows in parallel, setting B columns in parallel, estimating a number of set bits on R rows and columns, and implementing/providing a global value signal and setting and resetting all N 2 R bits, in parallel, in the cross-bar array when a reward signal arrives.
- FIG. 14 is a high level block diagram showing an information processing system 100 useful for implementing one embodiment of the present invention.
- the computer system includes one or more processors, such as processor 102.
- the processor 102 is connected to a communication infrastructure 104 (e.g., a communications bus, cross-over bar, or network).
- a communication infrastructure 104 e.g., a communications bus, cross-over bar, or network.
- the computer system can include a display interface 106 that forwards graphics, text, and other data from the communication infrastructure 104 (or from a frame buffer not shown) for display on a display unit 108.
- the computer system also includes a main memory 110, preferably random access memory (RAM), and may also include a secondary memory 112.
- the secondary memory 112 may include, for example, a hard disk drive 114 and/or a removable storage drive 116, representing, for example, a floppy disk drive, a magnetic tape drive, or an optical disk drive.
- the removable storage drive 116 reads from and/or writes to a removable storage unit 118 in a manner well known to those having ordinary skill in the art.
- Removable storage unit 118 represents, for example, a floppy disk, a compact disc, a magnetic tape, or an optical disk, etc. which is read by and written to by removable storage drive 116.
- the removable storage unit 118 includes a computer readable medium having stored therein computer software and/or data.
- the secondary memory 112 may include other similar means for allowing computer programs or other instructions to be loaded into the computer system.
- Such means may include, for example, a removable storage unit 120 and an interface 122.
- Examples of such means may include a program package and package interface (such as that found in video game devices), a removable memory chip (such as an EPROM, or PROM) and associated socket, and other removable storage units 120 and interfaces 122 which allow software and data to be transferred from the removable storage unit 120 to the computer system.
- the computer system may also include a communications interface 124.
- Communications interface 124 allows software and data to be transferred between the computer system and external devices. Examples of communications interface 124 may include a modem, a network interface (such as an Ethernet card), a communications port, or a PCMCIA slot and card, etc.
- Software and data transferred via communications interface 124 are in the form of signals which may be, for example, electronic, electromagnetic, optical, or other signals capable of being received by communications interface 124. These signals are provided to communications interface 124 via a communications path (i.e., channel) 126.
- This communications path 126 carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and/or other communications channels.
- computer program medium “computer usable medium,” and “computer readable medium” are used to generally refer to media such as main memory 110 and secondary memory 112, removable storage drive 116, and a hard disk installed in hard disk drive 114.
- Computer programs are stored in main memory 110 and/or secondary memory 112. Computer programs may also be received via
- Such computer programs when run, enable the computer system to perform the features of the present invention as discussed herein.
- the computer programs when run, enable the processor 102 to perform the features of the computer system. Accordingly, such computer programs represent controllers of the computer system.
Abstract
Description
Claims
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CA2817802A CA2817802C (en) | 2010-12-30 | 2011-10-18 | Electronic synapses for reinforcement learning |
KR1020137015687A KR101507671B1 (en) | 2010-12-30 | 2011-10-18 | Electronic synapses for reinforcement learning |
CN201180063280.5A CN103282919B (en) | 2010-12-30 | 2011-10-18 | The electronic synapse of intensified learning |
EP11772957.4A EP2641214B1 (en) | 2010-12-30 | 2011-10-18 | Electronic synapses for reinforcement learning |
JP2013546624A JP5907994B2 (en) | 2010-12-30 | 2011-10-18 | Systems, devices, and computer programs that include electronic synapses (electronic synapses for reinforcement learning) |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/982,505 | 2010-12-30 | ||
US12/982,505 US8892487B2 (en) | 2010-12-30 | 2010-12-30 | Electronic synapses for reinforcement learning |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2012089360A1 true WO2012089360A1 (en) | 2012-07-05 |
Family
ID=44852931
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2011/068183 WO2012089360A1 (en) | 2010-12-30 | 2011-10-18 | Electronic synapses for reinforcement learning |
Country Status (8)
Country | Link |
---|---|
US (1) | US8892487B2 (en) |
EP (1) | EP2641214B1 (en) |
JP (1) | JP5907994B2 (en) |
KR (1) | KR101507671B1 (en) |
CN (1) | CN103282919B (en) |
CA (1) | CA2817802C (en) |
TW (1) | TWI515670B (en) |
WO (1) | WO2012089360A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105684002A (en) * | 2013-10-28 | 2016-06-15 | 高通股份有限公司 | Methods and apparatus for tagging classes using supervised learning |
US9575142B2 (en) | 2014-11-28 | 2017-02-21 | International Business Machines Corporation | Sensor arrangement for position sensing |
US10282657B2 (en) | 2014-10-30 | 2019-05-07 | International Business Machines Corporation | Neuromorphic synapses |
Families Citing this family (47)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9405975B2 (en) | 2010-03-26 | 2016-08-02 | Brain Corporation | Apparatus and methods for pulse-code invariant object recognition |
US8712940B2 (en) * | 2011-05-31 | 2014-04-29 | International Business Machines Corporation | Structural plasticity in spiking neural networks with symmetric dual of an electronic neuron |
US8738554B2 (en) * | 2011-09-16 | 2014-05-27 | International Business Machines Corporation | Event-driven universal neural network circuit |
US8874498B2 (en) | 2011-09-16 | 2014-10-28 | International Business Machines Corporation | Unsupervised, supervised, and reinforced learning via spiking computation |
US8626684B2 (en) | 2011-12-14 | 2014-01-07 | International Business Machines Corporation | Multi-modal neural network for universal, online learning |
US9412041B1 (en) | 2012-06-29 | 2016-08-09 | Brain Corporation | Retinal apparatus and methods |
US9436908B2 (en) * | 2013-02-22 | 2016-09-06 | Qualcomm Technologies Inc. | Apparatus and methods for rate-modulated plasticity in a neuron network |
US9436909B2 (en) | 2013-06-19 | 2016-09-06 | Brain Corporation | Increased dynamic range artificial neuron network apparatus and methods |
US9552546B1 (en) | 2013-07-30 | 2017-01-24 | Brain Corporation | Apparatus and methods for efficacy balancing in a spiking neuron network |
KR20150016089A (en) * | 2013-08-02 | 2015-02-11 | 안병익 | Neural network computing apparatus and system, and method thereof |
US20150286925A1 (en) * | 2014-04-08 | 2015-10-08 | Qualcomm Incorporated | Modulating plasticity by global scalar values in a spiking neural network |
KR102366783B1 (en) * | 2014-07-07 | 2022-02-24 | 광주과학기술원 | Neuromorphic system operating method therefor |
KR102255309B1 (en) * | 2014-10-17 | 2021-05-25 | 삼성전자주식회사 | Artificial neuron semiconductor device with three dimension structure and artificial neuron semiconductor system using the same |
US9881349B1 (en) | 2014-10-24 | 2018-01-30 | Gopro, Inc. | Apparatus and methods for computerized object identification |
KR101701250B1 (en) * | 2015-08-03 | 2017-02-01 | 서울대학교산학협력단 | Multi-layered neuron array for deep belief network and neuron array operating method |
KR102446409B1 (en) | 2015-09-18 | 2022-09-22 | 삼성전자주식회사 | Method of fabricating synapse memory device |
US9767407B2 (en) | 2015-09-18 | 2017-09-19 | Samsung Electronics Co., Ltd. | Weighting device, neural network, and operating method of the weighting device |
CN107194462B (en) * | 2016-03-15 | 2020-05-19 | 清华大学 | Three-value neural network synapse array and neuromorphic computing network system using same |
CN107368888B (en) * | 2016-05-11 | 2021-05-25 | 上海磁宇信息科技有限公司 | Brain-like computing system and synapse thereof |
US11308383B2 (en) | 2016-05-17 | 2022-04-19 | Silicon Storage Technology, Inc. | Deep learning neural network classifier using non-volatile memory array |
US10713562B2 (en) * | 2016-06-18 | 2020-07-14 | International Business Machines Corporation | Neuromorphic memory circuit |
US10892330B2 (en) * | 2016-07-06 | 2021-01-12 | International Business Machines Corporation | FET based synapse network |
US10423878B2 (en) | 2016-09-07 | 2019-09-24 | International Business Machines Corporation | Artificial neuron apparatus |
US10748060B2 (en) * | 2016-10-14 | 2020-08-18 | Intel Corporation | Pre-synaptic learning using delayed causal updates |
US10090047B2 (en) * | 2016-11-09 | 2018-10-02 | International Business Machines Corporation | Memory cell structure |
CN108073982B (en) * | 2016-11-18 | 2020-01-03 | 上海磁宇信息科技有限公司 | Brain-like computing system |
CN108154225B (en) * | 2016-12-06 | 2021-09-03 | 上海磁宇信息科技有限公司 | Neural network chip using analog computation |
KR101997868B1 (en) | 2017-02-14 | 2019-07-09 | 서울대학교 산학협력단 | neuromorphic system, and memory device |
WO2018163005A1 (en) * | 2017-03-10 | 2018-09-13 | 株式会社半導体エネルギー研究所 | Touch panel system, electronic device, and semiconductor device |
US11222260B2 (en) * | 2017-03-22 | 2022-01-11 | Micron Technology, Inc. | Apparatuses and methods for operating neural networks |
US11087207B2 (en) | 2018-03-14 | 2021-08-10 | Silicon Storage Technology, Inc. | Decoders for analog neural memory in deep learning artificial neural network |
US10748630B2 (en) | 2017-11-29 | 2020-08-18 | Silicon Storage Technology, Inc. | High precision and highly efficient tuning mechanisms and algorithms for analog neuromorphic memory in artificial neural networks |
US10803943B2 (en) | 2017-11-29 | 2020-10-13 | Silicon Storage Technology, Inc. | Neural network classifier using array of four-gate non-volatile memory cells |
US10699779B2 (en) * | 2017-11-29 | 2020-06-30 | Silicon Storage Technology, Inc. | Neural network classifier using array of two-gate non-volatile memory cells |
JP6896678B2 (en) | 2018-07-02 | 2021-06-30 | 株式会社東芝 | Reinforcement learning system |
KR102618546B1 (en) * | 2018-09-03 | 2023-12-27 | 삼성전자주식회사 | 2-dimensional array based neuromorphic processor and operating method for the same |
US11500442B2 (en) | 2019-01-18 | 2022-11-15 | Silicon Storage Technology, Inc. | System for converting neuron current into neuron current-based time pulses in an analog neural memory in a deep learning artificial neural network |
US11023559B2 (en) | 2019-01-25 | 2021-06-01 | Microsemi Soc Corp. | Apparatus and method for combining analog neural net with FPGA routing in a monolithic integrated circuit |
US11270771B2 (en) | 2019-01-29 | 2022-03-08 | Silicon Storage Technology, Inc. | Neural network classifier using array of stacked gate non-volatile memory cells |
KR20200099252A (en) | 2019-02-13 | 2020-08-24 | 삼성전자주식회사 | A device for generating verification vector for verifying circuit design, circuit design system including the same and their reinforcement learning method |
US11270195B2 (en) | 2019-03-05 | 2022-03-08 | International Business Machines Corporation | Neuromorphic computing in dynamic random access memory |
US11423979B2 (en) | 2019-04-29 | 2022-08-23 | Silicon Storage Technology, Inc. | Decoding system and physical layout for analog neural memory in deep learning artificial neural network |
KR102425869B1 (en) * | 2019-12-09 | 2022-07-28 | 광주과학기술원 | CMOS-based crossbar deep learning accelerator |
JP7237891B2 (en) | 2020-07-15 | 2023-03-13 | ソフトバンク株式会社 | LEARNING EXECUTION DEVICE, PROGRAM, AND LEARNING EXECUTION METHOD |
US11790033B2 (en) * | 2020-09-16 | 2023-10-17 | International Business Machines Corporation | Accelerated Quasi-Newton methods on analog crossbar hardware |
EP4002220A1 (en) * | 2020-11-12 | 2022-05-25 | Commissariat à l'Energie Atomique et aux Energies Alternatives | Synapse circuit for three-factor learning |
CN113723602A (en) * | 2021-09-01 | 2021-11-30 | 哈尔滨工程大学 | Nerve synapse scheme of optical fiber structure |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080162391A1 (en) | 2006-12-29 | 2008-07-03 | Neurosciences Research Foundation, Inc. | Solving the distal reward problem through linkage of stdp and dopamine signaling |
WO2009113993A1 (en) | 2008-03-14 | 2009-09-17 | Hewlett-Packard Development Company, L.P. | Neuromorphic circuit |
Family Cites Families (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH03209553A (en) * | 1990-01-11 | 1991-09-12 | Mitsubishi Electric Corp | Neural circuit network device containing self-organizing function |
IT1244911B (en) | 1991-01-31 | 1994-09-13 | Texas Instruments Italia Spa | PHYSICALLY INSERABLE NEURONIC NETWORK ARCHITECTURE IN THE LEARNING PROCESS. |
US5274748A (en) | 1991-12-27 | 1993-12-28 | At&T Bell Laboratories | Electronic synapse circuit for artificial neural network |
JPH05210649A (en) | 1992-01-24 | 1993-08-20 | Mitsubishi Electric Corp | Neural network expressing device |
US5479579A (en) * | 1992-09-04 | 1995-12-26 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Cascaded VLSI neural network architecture for on-line learning |
US5355528A (en) * | 1992-10-13 | 1994-10-11 | The Regents Of The University Of California | Reprogrammable CNN and supercomputer |
GB9226192D0 (en) | 1992-12-16 | 1993-02-10 | Univ London | Neural network architecture |
JPH06290286A (en) * | 1993-04-06 | 1994-10-18 | Ricoh Co Ltd | Synapse circuit |
US5781702A (en) | 1995-06-07 | 1998-07-14 | Univ South Western | Hybrid chip-set architecture for artificial neural network system |
JP2001507772A (en) * | 1996-10-23 | 2001-06-12 | フォード、グローバル、テクノロジーズ、インコーポレーテッド | Misfire Detection System for Automotive Engine Including Recurrent Neuroprocessor Based on Bit Serial |
US6829598B2 (en) | 2000-10-02 | 2004-12-07 | Texas Instruments Incorporated | Method and apparatus for modeling a neural synapse function by utilizing a single conventional MOSFET |
DE60234900D1 (en) * | 2001-12-21 | 2010-02-11 | Ibm | Image scaling methods and circuits using neural networks |
EP1584004A4 (en) * | 2003-01-17 | 2007-10-24 | Francisco J Ayala | System and method for developing artificial intelligence |
US7430546B1 (en) | 2003-06-07 | 2008-09-30 | Roland Erwin Suri | Applications of an algorithm that mimics cortical processing |
TW594843B (en) * | 2003-07-30 | 2004-06-21 | Powerchip Semiconductor Corp | Management system of monitor wafers |
GB0506253D0 (en) | 2005-03-29 | 2005-05-04 | Univ Ulster | Electronic synapse device |
US7533071B2 (en) | 2005-06-28 | 2009-05-12 | Neurosciences Research Foundation, Inc. | Neural modeling and brain-based devices using special purpose processor |
US7599895B2 (en) | 2005-07-07 | 2009-10-06 | Knowm Tech, Llc | Methodology for the configuration and repair of unreliable switching elements |
US20090292661A1 (en) | 2008-05-21 | 2009-11-26 | Haas Alfred M | Compact Circuits and Adaptation Techniques for Implementing Adaptive Neurons and Synapses with Spike Timing Dependent Plasticity (STDP). |
GB0811057D0 (en) | 2008-06-17 | 2008-07-23 | Univ Ulster | Artificial neural network architecture |
US8463723B2 (en) | 2009-03-01 | 2013-06-11 | International Business Machines Corporation | Electronic synapse |
US8510244B2 (en) * | 2009-03-20 | 2013-08-13 | ISC8 Inc. | Apparatus comprising artificial neuronal assembly |
-
2010
- 2010-12-30 US US12/982,505 patent/US8892487B2/en active Active
-
2011
- 2011-10-18 JP JP2013546624A patent/JP5907994B2/en active Active
- 2011-10-18 KR KR1020137015687A patent/KR101507671B1/en active IP Right Grant
- 2011-10-18 EP EP11772957.4A patent/EP2641214B1/en active Active
- 2011-10-18 WO PCT/EP2011/068183 patent/WO2012089360A1/en active Application Filing
- 2011-10-18 CN CN201180063280.5A patent/CN103282919B/en active Active
- 2011-10-18 CA CA2817802A patent/CA2817802C/en active Active
- 2011-10-28 TW TW100139332A patent/TWI515670B/en active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080162391A1 (en) | 2006-12-29 | 2008-07-03 | Neurosciences Research Foundation, Inc. | Solving the distal reward problem through linkage of stdp and dopamine signaling |
WO2009113993A1 (en) | 2008-03-14 | 2009-09-17 | Hewlett-Packard Development Company, L.P. | Neuromorphic circuit |
Non-Patent Citations (3)
Title |
---|
FLORIAN R V: "Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity", INTERNET CITATION, 27 September 2006 (2006-09-27), pages 1 - 28, XP007914409, Retrieved from the Internet <URL:http://web.archive.org/web/20060927141830/http://www.coneural.org/flo rian/papers/06_modulated_STDP.pdf> [retrieved on 20100812] * |
JOHN V. ARTHUR ET AL: "Learning in Silicon: Timing is Everything", ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 18, 1 May 2006 (2006-05-01), XP055010017, ISBN: 978-0-26-223253-1, Retrieved from the Internet <URL:http://www.stanford.edu/group/brainsinsilicon/pdf/05_conf_NIPS_SiLearning.pdf> [retrieved on 20111019] * |
MISRA J ET AL: "Artificial neural networks in hardware: A survey of two decades of progress", NEUROCOMPUTING, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 74, no. 1-3, 1 December 2010 (2010-12-01), pages 239 - 255, XP027517200, ISSN: 0925-2312, [retrieved on 20101123], DOI: 10.1016/J.NEUCOM.2010.03.021 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105684002A (en) * | 2013-10-28 | 2016-06-15 | 高通股份有限公司 | Methods and apparatus for tagging classes using supervised learning |
CN105684002B (en) * | 2013-10-28 | 2018-07-20 | 高通股份有限公司 | For using supervised study to the tagged method and apparatus of type |
US10282657B2 (en) | 2014-10-30 | 2019-05-07 | International Business Machines Corporation | Neuromorphic synapses |
US9575142B2 (en) | 2014-11-28 | 2017-02-21 | International Business Machines Corporation | Sensor arrangement for position sensing |
Also Published As
Publication number | Publication date |
---|---|
EP2641214B1 (en) | 2017-06-14 |
CA2817802A1 (en) | 2012-07-05 |
CN103282919B (en) | 2016-02-17 |
KR101507671B1 (en) | 2015-03-31 |
TW201227545A (en) | 2012-07-01 |
US20140310220A1 (en) | 2014-10-16 |
EP2641214A1 (en) | 2013-09-25 |
CN103282919A (en) | 2013-09-04 |
JP2014504756A (en) | 2014-02-24 |
JP5907994B2 (en) | 2016-05-11 |
TWI515670B (en) | 2016-01-01 |
CA2817802C (en) | 2018-07-24 |
KR20130114183A (en) | 2013-10-16 |
US8892487B2 (en) | 2014-11-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CA2817802C (en) | Electronic synapses for reinforcement learning | |
US8812414B2 (en) | Low-power event-driven neural computing architecture in neural networks | |
US10885424B2 (en) | Structural plasticity in spiking neural networks with symmetric dual of an electronic neuron | |
US10628732B2 (en) | Reconfigurable and customizable general-purpose circuits for neural networks | |
US11238343B2 (en) | Scalable neural hardware for the noisy-OR model of Bayesian networks | |
US11074496B2 (en) | Providing transposable access to a synapse array using a recursive array layout | |
US8918351B2 (en) | Providing transposable access to a synapse array using column aggregation |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201180063280.5 Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11772957 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2013546624 Country of ref document: JP Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2817802 Country of ref document: CA |
|
ENP | Entry into the national phase |
Ref document number: 20137015687 Country of ref document: KR Kind code of ref document: A |
|
REEP | Request for entry into the european phase |
Ref document number: 2011772957 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2011772957 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |