OpenBMC introduction and porting guide
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Palanisamy, Saravanan
OpenBMC is an Open Source Software project started in an effort to create a secure, scalable, open source firmware code for BMC. Apart from the usual benefits arising from Open Source nature, OpenBMC brings in additional advantages like a.) state-of-the-art build system based on Yocto - an embedded linux distribution - which simplifies the process of building customized Linux, b.) Robust Managebility framework based on (4 pillars - REST, JSON, HTTPS, ODATAv4) RedFish, c.) Superior Modularity with D-bus IPC mechanism which is known for its well defined interfaces, d.) Ability to customize the code, e.) Support for IPMI, etc.
 Hi, hello, welcome to my presentation on open BMC introduction and putting guide. My name is Anupani Sonia, I work in Intel for past 12 years in a firmware and before it I have a total industry experience of 50 years. Okay, I think for today's topic is I give big introduction to open BMC and key features in open BMC like system to Yachto and do some production and I will make your first work on the community open BMC repo and I will try to do for you and then I will also make a customized build for new platform based on Intel pause and then it's built from the Intel BMC repo and then I will show you how properties in the platform are modified in editing manager and then it's like a demo for putting into the new platform. Okay, going to the introduction open BMC is a minus formation project and then it's a open source in here and its target is to produce a customizable open source form as tech for baseboard management controllers and most people know about BMC the description I have given here, you can read it, there is already one talk available last year, post and 2020 which you can read from in a couple. These are some of the key features of open BMC I am just highlighting here from the traditional BMC form and how it is different. It uses the set of modern technologies like for Linux distribution it uses Yachto and the long-distance choice is modern CPP Python and JSON and IPC mechanism is system D and the Vibras and then it supports for natively protocols like Redfish and IPM and Redfish is the feature which provides the security and scalability for better than IPM and IPM and you might be able to get it open for a bit of time. So this is not a complete key features list I am just highlighting it here for the key building blocks in open BMC. These are also some of the links to getting started all are pointed to the GitHub.com has open BMC.oxydepot or Viki link which you can read them. Coming back to Yachto, Yachto is like most aware of in the embedded Linux community. It opens a project that delivers set of tools to create a custom embedded Linux for your distribution. So for desktop server platforms there are a lot of Linux framework distribution, some of them are like Fed or Orbit, like I just said, a Debian like that. Whereas for the embedded Linux space there is most analysis and these kind of creating the client distribution is cumbersome. So Yachto come into the picture and it provides set of tools for that. So if you take an analogy of final Linux for this embedded image is cake which have multiple layers like this, bottom layer, middle layer and then top layer which have a creams and this is like chocolate like this. So similarly we can make analogy for the OIS like bottom layer is like BSPs and journals and applications and middleware, those kind of layers. We group the functionalities into the layers and we define the layers like this. And some of the prefixed, these layers are prefixed with Meta-iPhone in Yachto's terminology. Some example Meta-Pocky, Meta-S2.0 Meta-Phosphor. This Phosphor is anything in Phosphor in all VMCs like a reference implementation of some particular open VMCs on this. So all the available services in open VMC are under Meta-iPhone Phosphor you can find it out. And Pocky is a reference embedded Linux OS distribution given by Yachto which most of the people will inherit most of the open VMC builds will inherit for the default features. And BigBake is like a Python-based task scheduler which does the actual making process of this final cake. So it's like execution engine it goes through and forces all the layers and the recipes and then you find out the dependencies in what order it can combine with BigBake this individual layers and then how it can combine and create into the final wires. And let's do our first open VMC build from our community repo. So instructions are available here. So let me check what is the distribution of my machine is 18.0.4, 18.0.4 which is okay I think.0.0.4 is already available but it's not there. So let me clone the community repo which is GitHub.com. That's open VMCs. That's open VMCs.bit. Now it's cloned and then let me get inside this open VMC and then let me go into some standard release in this repo. So if I give GitHub it shows all the releases in this I am picking the release 2.9.0 and I am creating a branch out of it. It's called my first build. So then I am building image. So when I give dot space setup sourcing the setup it shows so many options available already in the community. So the community introduction reading dot empty page here is giving instructions for building the IBM Romulus image. Now I am giving instructions for building the WolfPos image which is Intel server platform. So this looks like this. This is one of the server platform. So this is the code for that WolfPos platform. So I am just saying setup my WolfPos platform and then keep the output in this directory called the build. So once I give this it will say like machines s2600 call in this like this. It will tell all this information. So that's it. Now you have to provide a very powerful feature of this web UI or content toaster which is useful to manage this big process and to understand the yacht also it's very helpful. So I can start this easy. I have to just give source toaster. There is some of the dependencies that you can see from here to install the necessary packages. So once I install like that toaster is ready to use. So I am starting a toaster in this particular address. This is the address in this server's machine and in this particular port. So it will take some 5 to 10 minutes first time. So I mean why once it's ready I can directly give a command to pick up the fcfmpossible image which will come back. So let's wait for some time. Okay now this toaster web server is started at this particular address 10.19 to 1.104 which I am going to this port and then across the meter. You can see the web UI for this build process. So let me start the build and then I can see in real time how it is going through. So the build is going to start. Okay our build is completed and I can see from the toaster environment. This is the image it's called build and then it's for the machine is 2600 wef and the project command and this I can see if I click this I can see what is the configuration of this water under the layers participating in this image. How many tasks it's run and task executed I can click and see what are all the tasks and recipes, packages and all the information is available here. Okay let me check the file actually built. It's found a build template which is the intd file and let me download the qmo one store and then I can now start this new little image using qmo. This is like how it really looked like when the first image boots which really awesome to see this in the qmo also in that way. So we built the image and we built the image and then now I am logging into the image. I can see what is the way it's really send the distribution of those things. Yeah this is all about qmo and coming to the system D concept it's mostly the way of it it's like I'm just going through fast. It's an IPC mechanism and it's a D bus and it have two type of buses system bus and the system bus and the system bus is for user space and the system bus is for connecting from the current space to user space communication. And these services are demons. System D is the one which is used in some Linux distributions which is to during the boot up which is taught the services and so all the services are demons is like equivalent to process. So these process have their unique name or unique number. So in this case client process have colon one seven zero two and service process have colon one four. So they are integrated using this D bus underline the connection which is like this they use to communicate to other. And three it is they need to know what is the target service name and what is the target object it is trying to reach and what is the names case or interface in which this object is residing. So four type of operations are four type of messages are possible to be done in the D bus. So like this plane can send a message in working method on this target object for which this way can either send a successful response which is called method return or it may give a error response which is called error. So if there is no activity there can be a synchronous event can be messages can be sent from the service process for any kind of events to notify who you are on subscribe to this. So these are the four messages available in this and this interface is the concept which is make one two namespaces in the C++ so that object parts are uniquely identified using this interface mechanism. The conventions are interfaces are separated by you know what elements are separated by dot whereas the object part is the naming place it's like point system or starting from the C++ or C++ like this is the entity manager which is like a runtime configuration application which passes the configuration files which are in the decent format and produces the best representation of the files on D bus using this namespace. So it also creates the system that doesn't file for persistence. So ideally what it has is now all the entities in the board like PSU or this base board or whatever the entities you name it. So those properties like our base board say have some 1090 sensors and the 1080C sensors like that for example and those sensors have cross holes and scaling and those kind of properties. So this kind of properties needs to be exposed by some entity so that can be consumed by some other entity. So entity manager does the job basically it identify what are all the properties of this board I mean this and all represented in this JSON format like PSU is represented in properties are encoded in JSON and similarly base board are encoded in JSON. So like it have all the base boards or PSUs in some common directory and then based on the true directory during the boot up it will identify which JSON is appropriate for this board and then load the properties from in a boot up. So these properties are exposed in the system D service object interfaces format and then that will be consumed by other applications like D bus sensors which is like independent application which monitors the sensors and it gets the values but to refer to the threshold value to raise alarm or what it does it subscribes to the entity manager and then for the particular interface from that only it get to know this sensor threshold value is this one. So by separating the data and mechanism and this entity manager does the job of forcing the JSON files and just exposing the system D names system D address space whereas this D bus sensor does the job of monitoring the sensors and then referring to the entity manager for the threshold and other properties. This clearly distinguishes for modularity and extensivity of this design. So this is what I mentioned just now so many JSON files are there which represent very different entities which entity manager will go through each JSON file and check whether it is matching with the current proof detected and if it is detecting load of the properties found in this design file and expose it as a system D properties which can be consumed by D bus sensor or whatever the system D compliant services. Now let me take you to the real example that I have a first platform this is the action with first platform configuration that the JSON file this is available in this community repo. So I have another clone of the first platform say which I want to create a BMC format stack or support for that open BMC format stack support. So the only difference between this first platform and my clone first platform my customer first platform is this ADC sensor at the index one. So this has a scaling factor of 5407 and this threshold is 3.647 that is my clone will pass have these values differences like this. So in this case how I can do about it is I will just copy this rule pass this WFTBase board of JSON file as forced from 2021.json file and directly modify this name and scale factor and values and then the probe statement also here it takes it takes work to go through this. So in JSON file there are two sections one is called this exposes which is grouped under this and then there is this probe section. So this probe is here nothing but the system in this interface is why is the open BMC product of true device in this interface if it counts this particular key and this value pair it's like a wild card start here WFT. So here it's like in the food if it is detecting your food device which have the product name ending with WFT it's basically assume there's a full pass board and this interface is exposed by another demo called the food device and that does the job of only monitoring the food devices and then identifying the food properties and then exposing it in the deepest. So as soon as it gets the notification this entity manager subscribes for this kind of notification and as soon as it gets it it's such as for this and it exposes it. So to try this we should have a prove the name ending with forced down. So if we have such a thing it will expose the properties like this. So let us see how it is doing it. Also to expose any wolf us this thing three levels of customization is needed. Level one is for the creating the Apple layers we can create a custom layers and then second thing is modifying the device tree in the kernel level. So usually if it is a ST 2 500 there is always much changes. So even if there can be examples which I'm not covering here and third one is exposing the user space which is like this. Yes and that's what I just mentioned. You can see the example of how to modify this layer like that documentation. So bblayers.com that sample is the one to describe the layers in the team to create your custom layer. So first of all you need to create a custom layer and then layer.com you need to create this layer with first icon forced down and then local.com. For example you need to create a machine called the first option forced down and here also in the layers to find the.com we will have to rename it as in the layers to find the.com. So once you are done and you can build this new image and you can build it from the Intel VMC repo also which is like another repo provided by Intel for the features which is said to be upstream and that is also you can try. This is the real one platform. It can find all the platforms under. So upon including and building you can find the WOT based board adjacent file and whereas forced WOT 21.json file. So the only difference between these two files is this particular sensor thing and then now let me modify that through entity correctly. IDMI to zero which is like a base board of proof. Is having the value like board.product name as s26002 WFT which is what referred to product.product name is ending with WFT that's why it is able to identify it as the board. Now let me modify this to force them value and let me see how it behaves. Let me edit this proof into s force them and I am using IP name to comment edit it. I have edited it and now it is showing like force them and then now if I go and see that properties it should reflect the new value. So as you can see now entity manager P3D3 now it is printing this property as P3D3 forced them to be 21. If I introspect this particular object using this procedure introspect command they can see this new scale 0.66638.88 like this. So basically like this we can put this create our custom JSON files and the custom proofs and then based on the proof and it can expose all the JSON properties. So yeah that's all sorry for taking too much of time. Thank you and question answers. Thanks. We'll be online shortly I think. Yes. So we are waiting for questions. Unfortunately we have just three minutes left. There was one question about architecture which is used on your platform. Could you reply this question? I saw that. Go ahead. Yeah I can try that. The question was about the architecture which is used on the platform which you are testing. Yes I can answer that question. Okay go ahead. Okay Intel uses ASP to ASP to 500 chip and yeah it is used in other company boards as well and this is based on the ARM architecture and single core and it's the most prevalent in the industry also and you can find support like that kernel device tree and all those things in that community the VMC repo also for this and the ASP documentation also available and it is also integrated into the kernel Linux kernels. Yeah that is about the Intel ASP to 500 and used in the old pass board and other platforms. Did you test different platforms or just only in Intel ones? I am not sure about other company platforms like IBM but it's like a dominant player in this VMC functionality space. Basically VMC need to do some manageability operations on the ARM core like video engine and interfacing with the BIOS like that. So this particular chip is very good in compacting all the functionalities and adding a small amount of other functions in there.
 Hi, hello, welcome to my presentation on open BMC introduction and putting guide.  My name is Anupani Sonia, I work in Intel for past 12 years in a firmware and before  it I have a total industry experience of 50 years.  Okay, I think for today's topic is I give big introduction to open BMC and key features  in open BMC like system to Yachto and do some production and I will make your first  work on the community open BMC repo and I will try to do for you and then I will also  make a customized build for new platform based on Intel pause and then it's built from the  Intel BMC repo and then I will show you how properties in the platform are modified in  editing manager and then it's like a demo for putting into the new platform.  Okay, going to the introduction open BMC is a minus formation project and then it's a  open source in here and its target is to produce a customizable open source form as  tech for baseboard management controllers and most people know about BMC the description  I have given here, you can read it, there is already one talk available last year, post  and 2020 which you can read from in a couple.  These are some of the key features of open BMC I am just highlighting here from the traditional  BMC form and how it is different.  It uses the set of modern technologies like for Linux distribution it uses Yachto and  the long-distance choice is modern CPP Python and JSON and IPC mechanism is system D and  the Vibras and then it supports for natively protocols like Redfish and IPM and Redfish  is the feature which provides the security and scalability for better than IPM and IPM  and you might be able to get it open for a bit of time.  So this is not a complete key features list I am just highlighting it here for the key  building blocks in open BMC.  These are also some of the links to getting started all are pointed to the GitHub.com  has open BMC.oxydepot or Viki link which you can read them.  Coming back to Yachto, Yachto is like most aware of in the embedded Linux community.  It opens a project that delivers set of tools to create a custom embedded Linux for your  distribution.  So for desktop server platforms there are a lot of Linux framework distribution, some  of them are like Fed or Orbit, like I just said, a Debian like that.  Whereas for the embedded Linux space there is most analysis and these kind of creating  the client distribution is cumbersome.  So Yachto come into the picture and it provides set of tools for that.  So if you take an analogy of final Linux for this embedded image is cake which have multiple  layers like this, bottom layer, middle layer and then top layer which have a creams and  this is like chocolate like this.  So similarly we can make analogy for the OIS like bottom layer is like BSPs and journals  and applications and middleware, those kind of layers.  We group the functionalities into the layers and we define the layers like this.  And some of the prefixed, these layers are prefixed with Meta-iPhone in Yachto's terminology.  Some example Meta-Pocky, Meta-S2.0 Meta-Phosphor.  This Phosphor is anything in Phosphor in all VMCs like a reference implementation of  some particular open VMCs on this.  So all the available services in open VMC are under Meta-iPhone Phosphor you can find  it out.  And Pocky is a reference embedded Linux OS distribution given by Yachto which most of  the people will inherit most of the open VMC builds will inherit for the default features.  And BigBake is like a Python-based task scheduler which does the actual making process of this  final cake.  So it's like execution engine it goes through and forces all the layers and the recipes  and then you find out the dependencies in what order it can combine with BigBake this  individual layers and then how it can combine and create into the final wires.  And let's do our first open VMC build from our community repo.  So instructions are available here.  So let me check what is the distribution of my machine is 18.0.4, 18.0.4 which is okay  I think.0.0.4 is already available but it's not there.  So let me clone the community repo which is GitHub.com.  That's open VMCs.  That's open VMCs.bit.  Now it's cloned and then let me get inside this open VMC and then let me go into some  standard release in this repo.  So if I give GitHub it shows all the releases in this I am picking the release 2.9.0 and  I am creating a branch out of it.  It's called my first build.  So then I am building image.  So when I give dot space setup sourcing the setup it shows so many options available already  in the community.  So the community introduction reading dot empty page here is giving instructions for  building the IBM Romulus image.  Now I am giving instructions for building the WolfPos image which is Intel server platform.  So this looks like this.  This is one of the server platform.  So this is the code for that WolfPos platform.  So I am just saying setup my WolfPos platform and then keep the output in this directory  called the build.  So once I give this it will say like machines s2600 call in this like this.  It will tell all this information.  So that's it.  Now you have to provide a very powerful feature of this web UI or content toaster which is  useful to manage this big process and to understand the yacht also it's very helpful.  So I can start this easy.  I have to just give source toaster.  There is some of the dependencies that you can see from here to install the necessary  packages.  So once I install like that toaster is ready to use.  So I am starting a toaster in this particular address.  This is the address in this server's machine and in this particular port.  So it will take some 5 to 10 minutes first time.  So I mean why once it's ready I can directly give a command to pick up the fcfmpossible  image which will come back.  So let's wait for some time.  Okay now this toaster web server is started at this particular address 10.19 to 1.104  which I am going to this port and then across the meter.  You can see the web UI for this build process.  So let me start the build and then I can see in real time how it is going through.  So the build is going to start.  Okay our build is completed and I can see from the toaster environment.  This is the image it's called build and then it's for the machine is 2600 wef and the project  command and this I can see if I click this I can see what is the configuration of this  water under the layers participating in this image.  How many tasks it's run and task executed I can click and see what are all the tasks  and recipes, packages and all the information is available here.  Okay let me check the file actually built.  It's found a build template which is the intd file and let me download the qmo one store  and then I can now start this new little image using qmo.  This is like how it really looked like when the first image boots which really awesome  to see this in the qmo also in that way.  So we built the image and we built the image and then now I am logging into the image.  I can see what is the way it's really send the distribution of those things.  Yeah this is all about qmo and coming to the system D concept it's mostly the way of  it it's like I'm just going through fast.  It's an IPC mechanism and it's a D bus and it have two type of buses system bus and  the system bus and the system bus is for user space and the system bus is for connecting  from the current space to user space communication.  And these services are demons.  System D is the one which is used in some Linux distributions which is to during the  boot up which is taught the services and so all the services are demons is like equivalent  to process.  So these process have their unique name or unique number.  So in this case client process have colon one seven zero two and service process have  colon one four.  So they are integrated using this D bus underline the connection which is like this they  use to communicate to other.  And three it is they need to know what is the target service name and what is the target  object it is trying to reach and what is the names case or interface in which this object  is residing.  So four type of operations are four type of messages are possible to be done in the D  bus.  So like this plane can send a message in working method on this target object for which this  way can either send a successful response which is called method return or it may give  a error response which is called error.  So if there is no activity there can be a synchronous event can be messages can be sent  from the service process for any kind of events to notify who you are on subscribe to this.  So these are the four messages available in this and this interface is the concept which  is make one two namespaces in the C++ so that object parts are uniquely identified using  this interface mechanism.  The conventions are interfaces are separated by you know what elements are separated by  dot whereas the object part is the naming place it's like point system or starting from  the C++ or C++ like this is the entity manager which is like a runtime configuration application  which passes the configuration files which are in the decent format and produces the  best representation of the files on D bus using this namespace.  So it also creates the system that doesn't file for persistence.  So ideally what it has is now all the entities in the board like PSU or this base board or  whatever the entities you name it.  So those properties like our base board say have some 1090 sensors and the 1080C sensors  like that for example and those sensors have cross holes and scaling and those kind of  properties.  So this kind of properties needs to be exposed by some entity so that can be consumed by  some other entity.  So entity manager does the job basically it identify what are all the properties of this  board I mean this and all represented in this JSON format like PSU is represented in properties  are encoded in JSON and similarly base board are encoded in JSON.  So like it have all the base boards or PSUs in some common directory and then based on  the true directory during the boot up it will identify which JSON is appropriate for this  board and then load the properties from in a boot up.  So these properties are exposed in the system D service object interfaces format and then  that will be consumed by other applications like D bus sensors which is like independent  application which monitors the sensors and it gets the values but to refer to the threshold  value to raise alarm or what it does it subscribes to the entity manager and then for the particular  interface from that only it get to know this sensor threshold value is this one.  So by separating the data and mechanism and this entity manager does the job of forcing  the JSON files and just exposing the system D names system D address space whereas this  D bus sensor does the job of monitoring the sensors and then referring to the entity manager  for the threshold and other properties.  This clearly distinguishes for modularity and extensivity of this design.  So this is what I mentioned just now so many JSON files are there which represent very  different entities which entity manager will go through each JSON file and check whether  it is matching with the current proof detected and if it is detecting load of the properties  found in this design file and expose it as a system D properties which can be consumed  by D bus sensor or whatever the system D compliant services.  Now let me take you to the real example that I have a first platform this is the action  with first platform configuration that the JSON file this is available in this community  repo.  So I have another clone of the first platform say which I want to create a BMC format stack  or support for that open BMC format stack support.  So the only difference between this first platform and my clone first platform my customer  first platform is this ADC sensor at the index one.  So this has a scaling factor of 5407 and this threshold is 3.647 that is my clone will pass  have these values differences like this.  So in this case how I can do about it is I will just copy this rule pass this WFTBase  board of JSON file as forced from 2021.json file and directly modify this name and scale  factor and values and then the probe statement also here it takes it takes work to go through  this.  So in JSON file there are two sections one is called this exposes which is grouped under  this and then there is this probe section.  So this probe is here nothing but the system in this interface is why is the open BMC product  of true device in this interface if it counts this particular key and this value pair it's  like a wild card start here WFT.  So here it's like in the food if it is detecting your food device which have the product name  ending with WFT it's basically assume there's a full pass board and this interface is exposed  by another demo called the food device and that does the job of only monitoring the food  devices and then identifying the food properties and then exposing it in the deepest.  So as soon as it gets the notification this entity manager subscribes for this kind of  notification and as soon as it gets it it's such as for this and it exposes it.  So to try this we should have a prove the name ending with forced down.  So if we have such a thing it will expose the properties like this.  So let us see how it is doing it.  Also to expose any wolf us this thing three levels of customization is needed.  Level one is for the creating the Apple layers we can create a custom layers and then second  thing is modifying the device tree in the kernel level.  So usually if it is a ST 2 500 there is always much changes.  So even if there can be examples which I'm not covering here and third one is exposing  the user space which is like this.  Yes and that's what I just mentioned.  You can see the example of how to modify this layer like that documentation.  So bblayers.com that sample is the one to describe the layers in the team to create  your custom layer.  So first of all you need to create a custom layer and then layer.com you need to create  this layer with first icon forced down and then local.com.  For example you need to create a machine called the first option forced down and here also  in the layers to find the.com we will have to rename it as in the layers to find the .com.  So once you are done and you can build this new image and you can build it from the Intel  VMC repo also which is like another repo provided by Intel for the features which is  said to be upstream and that is also you can try.  This is the real one platform.  It can find all the platforms under.  So upon including and building you can find the WOT based board adjacent file and whereas  forced WOT 21.json file.  So the only difference between these two files is this particular sensor thing and then now  let me modify that through entity correctly.  IDMI to zero which is like a base board of proof.  Is having the value like board.product name as s26002 WFT which is what referred to product.product  name is ending with WFT that's why it is able to identify it as the board.  Now let me modify this to force them value and let me see how it behaves.  Let me edit this proof into s force them and I am using IP name to comment edit it.  I have edited it and now it is showing like force them and then now if I go and see that  properties it should reflect the new value.  So as you can see now entity manager P3D3 now it is printing this property as P3D3 forced  them to be 21.  If I introspect this particular object using this procedure introspect command they can  see this new scale 0.66638.88 like this.  So basically like this we can put this create our custom JSON files and the custom proofs  and then based on the proof and it can expose all the JSON properties.  So yeah that's all sorry for taking too much of time.  Thank you and question answers.  Thanks.  We'll be online shortly I think.  Yes.  So we are waiting for questions.  Unfortunately we have just three minutes left.  There was one question about architecture which is used on your platform.  Could you reply this question?  I saw that.  Go ahead.  Yeah I can try that.  The question was about the architecture which is used on the platform which you are testing.  Yes I can answer that question.  Okay go ahead.  Okay Intel uses ASP to ASP to 500 chip and yeah it is used in other company boards as  well and this is based on the ARM architecture and single core and it's the most prevalent  in the industry also and you can find support like that kernel device tree and all those  things in that community the VMC repo also for this and the ASP documentation also available  and it is also integrated into the kernel Linux kernels.  Yeah that is about the Intel ASP to 500 and used in the old pass board and other platforms.  Did you test different platforms or just only in Intel ones?  I am not sure about other company platforms like IBM but it's like a dominant player in  this VMC functionality space.  Basically VMC need to do some manageability operations on the ARM core like video engine  and interfacing with the BIOS like that.  So this particular chip is very good in compacting all the functionalities and adding a small  amount of other functions in there.
firmware_oiapg_0_131.png firmware_oiapg_132_7403.png firmware_oiapg_7404_7583.png firmware_oiapg_7584_8183.png firmware_oiapg_8184_8603.png firmware_oiapg_8604_9047.png firmware_oiapg_9048_10067.png firmware_oiapg_10068_12971.png firmware_oiapg_12972_13343.png firmware_oiapg_13344_13811.png firmware_oiapg_13812_14279.png firmware_oiapg_14280_14507.png firmware_oiapg_14508_14615.png firmware_oiapg_14616_14663.png firmware_oiapg_14664_14867.png firmware_oiapg_14868_22067.png firmware_oiapg_22068_22871.png firmware_oiapg_22872_26579.png firmware_oiapg_26580_27335.png firmware_oiapg_27336_29363.png firmware_oiapg_29364_29411.png firmware_oiapg_29412_29507.png firmware_oiapg_29508_29879.png firmware_oiapg_29880_30299.png firmware_oiapg_30300_32387.png firmware_oiapg_32388_32435.png firmware_oiapg_32436_36002.png