Q:

Android RxJava/Kotlin - map large set of data to network calls

I am trying to implement an Android app which needs to obtain a big amount of data from a backend service and save it to a db to later work on it.
The below code describes the process: 
itemsService
    .getAllItemIds() //This returns Single<List<Int>> from backend
    .subscribeOn(Schedulers.io())
    .subscribe({
        Observable.fromIterable(it)
        .map({
            itemsService
            .getItemById(it) //This gets one item details from backend
            .subscribeOn(Schedulers.io())
            .observeOn(AndroidSchedulers.mainThread())
            .subscribe({
                //Add item details to db
            }, {
                //Some error
            })
        })
    }, {
        //Some error
    })

I obtain a list of ids and then map each of these ids to a network call to obtain the full object.
This works for a test set of, say, 10 items, but the production set contains over 50 000 ids. It works initially, saving the items, but around 5-10% it grinds to a halt and the app dies.
I assume the reason here would be that Rx keeps the reference between the source and the mapped value.
My question is: is there a way to "pool" the source emissions to, let's say, 10 at a time? Or maybe there is some other mechanism I am not aware of?

A:

You didn't mention what exactly "grinds to a halt" means, but it makes sense that you will get out of memory in real case of 50,000 items, cause you will basically try to create 50,000 threads at once to fetch each items details.
moreover, instead of chaining Observables using operators, you're creating nested chains at subscribe/map, you can read here why you shouldn't.
regarding limiting the work to 10 at a time, there is an flatMap overload for that, at the end it might look something like this:
itemsService
    .getAllItemIds() //This returns List<Int> from backend
    .flatMapIterable { t -> t }
    .flatMap({
        itemsService
                .getItemById(it) //This gets one item details from backend
                .subscribeOn(Schedulers.io())
    }, 10) //limit flat map parallelism by desired value 
    .subscribeOn(Schedulers.io())
    .observeOn(AndroidSchedulers.mainThread())
    .subscribe({
        //Add item details to db
    }, {
        //Some error
    })

