Memory System

This page documents Kermit memory system internals.

The OMAP35x Technical Reference Manual provides information about the SonicsMX® interconnect used in Kermit ES1. Datasheets for some Microchip PIC32 microcontrollers (e.g. PIC32MZ family) provide information about the SonicsSX® interconnect used in Kermit ES2+.

Glossary

• SMX

Abbreviation of SonicsMX®.

• SSX

Abbreviation of SonicsSX®.

• Module

Any device connected to the memory system, such as ARM cores, DMA controllers, LPDDR2... Modules can be initiators, targets or both.

• Initiator

A module that can initiate read and write requests to the interconnect (e.g. ARM cores, DMA, ...).

• Target

A module that can only respond to requests from the interconnect. Targets may also be able to generate out-of-band signals such as interrupts.

• Agent

The connection between a module's port and the interconnect. If a module has multiple ports (e.g. an Initiator and Target module), each port is connected to the interconnect by a different agent (e.g. one IA and one TA).

• TA

Short for Target Agent. The agent connecting an Initiator module to the interconnect.

• IA

Short for Initiator Agent. The agent connecting a Target module to the interconnect.

• OCP

Open-core Protocol, a standard point-to-point protocol allowing communication between a master and a slave port.

• OCP Master Port

A port that can generate OCP commands. Initiators always include at least one master port.

• OCP Slave Port

A port that can respond to OCP commands. Targets always include one slave port.

• Interconnect

Logic device enabling connection between initiator and target modules connected to it.

• Out-of-band error

OCP signal associated to a device error-reporting scheme. This is opposed to an in-band error, which is associated to the protocol's error-reporting scheme. Out-of-band errors are also known as sideband errors.

Generalities

The bus fabric of Kermit is made of several SonicsSX® interconnects on which peripherals are connected.

All communication is done using the OCP protocol; however, adapters (e.g., AXI2OCP for Cortex-A9) may be present at Agent level.

There are three revisions of the memory system layout:

• Original layout for Kermit1.0 ES1 - uses the SMX interconnect (as found in BeagleBoard SoC) instead of SSX
• Second revision for Kermit1.0 ES2 - very close to the final revision, uses the SSX interconnect
• Third and final revision for Kermit1.0 ES3 and later

In the following sections, you may see mentions of register names such as BEBT and SBEBT. Unless otherwise specified, a register prefixed with S is merely the ARM Secure variant of the register e.g. SBEBT is BEBT for ARM Secure. Everything that applies to a register should apply to its Secure variant, except that the register is updated by Secure bus transactions, instead of Non-Secure bus transactions.

Signals

MReqInfo

MReqInfo is 16 bits wide in Kermit.

Bits <6:0> contain the Bus Master ID of initiator. This is sometimes used for access checks, or to identify transaction initiator when a bus error occurs.

Interconnect registers

There are register blocks associated to each interconnect in the system. These blocks are always 1KiB (0x400 bytes), and the registers inside them are always 64-bit.

Location

Address	Name	UIID
0xEC000000	Main Xbar	0
0xEC100000	Center Xbar	1
0xEC200000	Video Xbar	2
0xEC300000	IFTU Xbar	3
0xEC400000	VIP Xbar	4
0xEC500000	Debug Bus	5
0xEC600000	DMAC Xbar	6
0xED000000	IO Slave Bus	8
0xED100000	IO Master Bus	9
0xED200000	Pervasive Bus	10
0xED300000	Misc IO Bus	11
0xED400000	North IO Master Bus	12
0xED500000	Audio Bus	13
0xED600000	North IO Slave Bus	14
0xED700000	SRC Bus	15
0xED800000	GPU Register Bus	16
0xED900000	IFTU Register Bus	17
0xEE000000	Camera Slave Bus	24
0xEE100000	Camera Master Bus	25
0xEE200000	USB Slave Bus	26
0xEE300000	USB Master Bus	27
0xEE400000	SubLCD Slave Bus	28
0xEE500000	SubLCD Master Bus	29
0xEE600000	Camera Register Bus	30
0xE00C0000	Cry/SBL Bus	48

The UIID column indicates the Unique Interconnect ID. Bit N of BEBT register indicates an error is pending on interconnect with UUID N.

Each interconnect register group is divided as follows:

Address mask	Sub-block	Description
0xEoo00000	?	?Sideband Interconnect?
0xEoo1xx00	?	IA-related (Only present if IA exists at 0xEoo4xx00)
0xEoo30000	?	?
0xEoo30400	?	?
0xEoo3C000	?	?
0xEoo4xx00	IA	Initiator Agent - one block for each initiator on the interconnect. Cross-interconnect IAs usually start at offset 0x8000.
0xEoo5xx00	TA	Target Agent - one block for each target on the interconnect. Cross-interconnect TAs usually start at offset 0x8000.
0xEoo6xx00	PM	Protection Mechanism - used to filter access to specific targets. If present, register block at 0xEoo6xx00 protects TA 0xEoo5xx00.
0xEoo6C000	Interconnect PM	Protection Mechanism ?for the interconnect's RT?
0xEoo70000	RT	Register Target - used to interact with the interconnect itself.

Note: Cry/SBL Bus does not follow this scheme.

Agent registers

IA and TA register blocks share a similar layout.

(List of physical address for all these interfaces can be found on Physical Memory page - names start with SSX_ - will be migrated here at some point)

Agent registers description

Name	Offset	Usage/Notes
COMPONENT	0x0	Bits [31:16] contain component code (CODE) Bits [15:0] contain component revision (REV) ?what is component referring to? SSX?
CORE	0x18	Bits [47:32] contain vendor code (VENDOR_CODE) Bits [31:16] contain core code (CORE_CODE) Bits [15:0] contain revision code (REV_CODE)
AGENT_CONTROL	0x20	?
AGENT_STATUS	0x28	Bit 0x0100_0000 indicates an error
AGENT_ERROR_LOG	0x58
AGENT_ERROR_LOG_ADDR	0x60
?	0x100	Seen values: 1

Register Target

Hosts configuration and status registers of the interconnect itself.

RT registers description

Name	Offset	Usage/Notes
COMPONENT	0x0	See Agent registers
NETWORK	0x10	Bits 63:48 (63:32?) hold the Unique Interconnect ID
INITID_READBACK	0x70	Bits <7:0> contain the Initiator ID of the reader
NETWORK_CONTROL	0x78	Bits <10:8> contain TIMEOUT_BASE

Note: the initiator ID is an interconnect-level concept, and unrelated to the Bus Master ID. ?Every IA has its own IID?

Sideband Interconnect

Each interconnect hosts a register block called Sideband Interconnect, used for error reporting.

SI registers description

Name	Offset	Usage/Notes
SI_FLAG_STATUS_0	0x110	Sideband Interconnect error status 0
SI_FLAG_STATUS_1	0x130	Sideband Interconnect error status 1
SI_FLAG_STATUS_2	0x150	Sideband Interconnect error status 2
SI_FLAG_STATUS_3	0x170	Sideband Interconnect error status 3
SI_FLAG_STATUS_4	0x190	Sideband Interconnect error status 4

+ many registers not documented in OMAP35x TRM

Protection Mechanism

Each Target Agent may be equipped with a Firewall (also called Protection Mechanism), which can be used to restrict or filter access to it based on various criteria.

PM registers location

Base address	Protected TA
0xEC060000?	Spad_32KiB?
0xEC060400	LPDDR0 (Main LPDDR2 + Controller I/F)
0xEC160000?	Compati_2MiB?
0xEC160400?	DevNull?
0xEC260000?	Spad_128KiB?
0xEC560000?	SDIO0?
0xEC560400?	SDIO1?
0xEC560800?	LPDDR1? (TOOL LPDDR2 + Controller I/F)

Bus Errors

When a bus error occurs, an interrupt is delivered to ARM Secure (Interrupt ID 0x21=33) or Non-Secure (Interrupt ID 0x20=32). The OS considers all bus errors to be fatal. The interrupt handlers installed for these IDs perform a register dump then stop the system.

It appears that device-initiated bus errors (e.g. DMAC) are always routed to Non-Secure interrupt. There may be other rules to consider (e.g. some register to set whether a device is Secure/Non-Secure).

It is unknown whether or not CMeP can receive bus error interrupts or how CMeP fits in the memory system.

There are two kind of bus errors: Internal Bus error and Target Device error. An Internal Bus error occurs when the memory system fails to deliver a request to a target device - for example, trying to access non-existent memory. A Target Device error occurs when a device successfully receives a request but is unable to handle it - for example, accessing the non-existent part of a device's memory.

Bus Error Attribute

The bus error attribute is a 32-bit value that can be recovered in a per-device MMIO register along with the bus error address.

The attribute holds multiple informations: which agent was the bus master when the error occurred, what bus command was ongoing, and, when available, the reason of the bus error.

Bus error attribute meaning

Bitmask	Name	Notes
0xFF80_0000	MReqInfo<15:7>	Bits <15:7> of MReqInfo signal
0x003F_0000	master	Bus master
0x0000_0700	cmd	MCmd signal
0x0000_0008	Burst Access Error
0x0000_0004	Register Permission Error
0x0000_0001	Address Hole Error	If set, the error occurred because the address was invalid.

Sometimes, this register is not present but separated in multiple (e.g., in REGBUS).

Note that for some devices (Spad32K, Spad128K, Compati SRAM), the only valid attribute is 0x1, indicating an invalid address.

To decode the meaning of master and cmd, shift them by 16 and 8 respectively to obtain a value between 0-63/0-7 and use the following tables:

MCmd table

Value	Command
0	Idle
1	Write
2	Read
3	ReadEx
4	ReadLinked
5	WriteNonPost
6	WriteConditional
7	Broadcast

Bus Master IDs (master decoding table)

Value	Command
0	Reserved
1	ARM Core0
2	ARM Core1
3	ARM Core2
4	ARM Core3
5	ARM L2
6	Reserved
7	Reserved
8	Reserved
9	Reserved
10	Reserved
11	Reserved
12	Reserved
13	Reserved
14	Reserved
15	Reserved
16	DMAC0
17	DMAC1
18	DMAC2
19	DMAC3
20	DMAC4
21	DMAC5
22	Reserved (Bigmac?)
23	GPU
24	Venezia
25	VIP vdpd
26	VIP vdpm
27	VIP bap
28	Reserved
29	IFTU0a
30	IFTU0b
31	IFTU1a
32	IFTU1b
33	IFTU2
34	Reserved (Compati port0 in ES1 SceKernelBusError)
35	Reserved (Compati port0 in ES1 SceKernelBusError)
36	USB1 Host EHCI
37	USB1 Host OCHI
38	Reserved (CMeP?)
39	Sensor in 0
40	Sensor in 1
41	Reserved (Sensor in 2 in ES1 SceKernelBusError)
42	LCD DMAC
43	Performance Monitor
44	USB2 Device DMAC
45	Sub LCD
46	DMAC6
47	USB0 Host EHCI
48	USB0 Host OHCI
49	USB2 Host EHCI
50	USB2 Host OHCI
51	USB0 Device DMAC
52	Reserved (USB1 Device DMAC1 in ES1/ES2)
53	Reserved (USB1 Device DMAC2 in ES1/ES2)
54	Reserved (USB1 Device DMAC3 in ES1/ES2)
55	USB1 Device DMAC
56	Reserved (USB2 Device DMAC1 in ES1/ES2)
57	Reserved (USB2 Device DMAC2 in ES1/ES2)
58	Reserved (USB2 Device DMAC3 in ES1/ES2)
59	SD/HSMMC0
60	SD/HSMMC1
61	SD/HSMMC2
62	SD/HSMMC3
63	Memory Stick

Clear TA error

In old System Software versions (e.g. 0.920 - pre-ES3, so this may no longer be valid), there exists inside the bus error module a function named BusErrorClearTA which works the following way.

First, choose a bus/XBar that will serve as start point (MainXBar for ES2). Second, recursively build a list of all TAs connected to this start point. Repeat the process for all TAs that are busses or XBars. Third, walk the obtained tree. Check the STATUS of all TAs. If an error is present, repeat the procedure recursively (if bus/XBar) then clear the error.

To detect an error, check if bit 0x01000_0000ull is present in STATUS To clear the error, simply write 0x01000_0000ull to STATUS.

NOTE: old System Software versions do not have informations about IAs, but only TAs. This could explain the routine's name, but also means this procedure may also work/be needed on IA side.

Miscellaneous

The memory system is able to distinguish if a transaction is originating from ARM Secure state or Non-Secure state because ARM Cortex-A9 processors with Security Extensions have a bit indicating whether the access is Secure or Non-Secure added to all memory system transactions.

Attempting to perform a DMAC memcpy() from Secure to Non-Secure LPDDR0 region results in a NS bus error. The current hypothesis is that all devices on the Kermit bus diagram that have an IA are treated as ARM NS.