I would like to report progress from a long running PLINQ query.

I can't really find any native LINQ method that allows me to do this (as was [implemented](https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/how-to-cancel-a-plinq-query) for [cancellation](https://learn.microsoft.com/en-us/dotnet/api/system.linq.parallelenumerable.withcancellation?view=netframework-4.7.2)). 

I have read [this article](http://blog.functionalfun.net/2008/07/reporting-progress-during-linq-queries.html) that shows a neat extension function for a regular serialized query. 

I have been testing the behavior using the below code. 

```c#
var progress = new BehaviorSubject<int>(0);
DateTime start = DateTime.Now;
progress.Subscribe(x => { Console.WriteLine(x); });
Enumerable.Range(1,1000000)
    //.WithProgressReporting(i => progress.OnNext(i)) //Beginning Progress
    .AsParallel()
    .AsOrdered()
    //.WithProgressReporting(i => progress.OnNext(i)) //Middle Progress reporting
    .Select(v => { Thread.Sleep(1); return v * v; })
    //.WithProgressReporting(i => progress.OnNext(i)) //End Progress Reporting
    .ToList();
Console.WriteLine("Completed in: " + (DateTime.Now - start).TotalSeconds + " seconds");

```

Edit:  

Reporting progress from the **middle** using the `IEnumerable<T>` extension removes the parallelism. 

Reporting from the **end** does not report any progress while the parallel calculations are being computed then quickly reports all the progress at the very end. I assume this is progress of compiling of the results from the parallel computation into a list. 

I originally thought the progress reporting from **beginning** was causing the the LINQ to run unparallelized. After sleeping on this, and reading comments from [Peter Duniho](https://stackoverflow.com/questions/55584078/how-can-i-report-progress-from-a-plinq-query#comment97866126_55584078), I see that it is actually working as parallel, but I am getting so many progress reports that handling so many is causing my test/application to slow significantly. 

Is there a way that is parallel/thread safe to report progress from a PLINQ in increments that allow a user to know progress is being made without having a significant impact on the method runtime?
['https://Stackoverflow.com/questions/55584078', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/7371817/']
This answer might not be as elegant, but it gets the job done.

When using PLINQ, there are multiple threads processing your collection, so using those threads to **report** progress results in multiple (and out-of-order) progress reports like 0% 1% 5% 4% 3% etc...

Instead, we can use those multiple threads to **update** a shared variable storing the progress. In my example, it's a local variable `completed`. We then spawn another thread using `Task.Run()` to report on that progress variable at 0.5s intervals.

Extension class:

```
static class Extensions
    public static ParallelQuery<T> WithProgressReporting<T>(this ParallelQuery<T> sequence, Action increment)
    {
        return sequence.Select(x =>
        {
            increment?.Invoke();
            return x;
        });
    }
}

```

Code:

```
static void Main(string[] args)
    {
        long completed = 0;
        Task.Run(() =>
        {
            while (completed < 100000)
            {
                Console.WriteLine((completed * 100 / 100000) + "%");
                Thread.Sleep(500);
            }
        });
        DateTime start = DateTime.Now;
        var output = Enumerable.Range(1, 100000)
            .AsParallel()
            .WithProgressReporting(()=>Interlocked.Increment(ref completed))
            .Select(v => { Thread.Sleep(1); return v * v; })
            .ToList();
        Console.WriteLine("Completed in: " + (DateTime.Now - start).TotalSeconds + " seconds");
        Console.ReadKey();
    }

```
This extension can be positioned either at the start or at the end of the LINQ query. If positioned at the start will start reporting progress immediately, but will erroneously report 100% before the job is done. If positioned at the end will report accurately the end of the query, but will delay reporting progress until the first item of the source has completed.

```
public static ParallelQuery<TSource> WithProgressReporting<TSource>(
    this ParallelQuery<TSource> source,
    long itemsCount, IProgress<double> progress)
{
    int countShared = 0;
    return source.Select(item =>
    {
        int countLocal = Interlocked.Increment(ref countShared);
        progress.Report(countLocal / (double)itemsCount);
        return item;
    });
}

```

Usage example:

```
// The Progress captures the current SynchronizationContext at construction.
var progress = new Progress<double>(); 
progress.ProgressChanged += (object sender, double e) =>
{
    Console.WriteLine($"Progress: {e:0%}");
};

var numbers = Enumerable.Range(1, 10);

var sum = numbers
    .AsParallel()
    .WithDegreeOfParallelism(3)
    .WithMergeOptions(ParallelMergeOptions.NotBuffered)
    .Select(n => { Thread.Sleep(500); return n; }) // Simulate some heavy computation
    .WithProgressReporting(10, progress) // <--- the extension method
    .Sum();

Console.WriteLine($"Sum: {sum}");

```

Output:

[![Query output](https://i.stack.imgur.com/gZjY9.png)](https://i.stack.imgur.com/gZjY9.png)

There is some jumping because sometimes the worker threads are preempting each other.

The [`System.Progress<T>`](https://learn.microsoft.com/en-us/dotnet/api/system.progress-1?view=netframework-4.7.2) class has the nice feature that invokes the `ProgressChanged` event at the captured context (usually the UI thread), so the UI controls can be updated safely. On the other hand in a Console Application the event is invoked on the ThreadPool, that will probably be fully utilized by the parallel query, so the event will fire with some delay (the ThreadPool spawns new threads every 500 msec). This is the reason I limited the parallelism to 3 in the example, to keep a free thread for the reporting of the progress (I have a quad core machine).