Q:

Override public virtual function with private base function?

Let's consider two classes A and B with the following interface:
class A {
public:
    virtual void start() {} //default implementation does nothing
};

class B {
public:
    void start() {/*do some stuff*/}
};

and then a third class which inherits from both, A publicly because it implements this "interface", and B privately because that's implementation detail.
However, in this specific implementation, start() only has to contain a call to B::start(). So I thought I could use a shortcut and do the following:
class C: public A, private B {
public:
    using B::start;
};

and be done with it, but apparently it doesn't work. So I get using private base function doesn't work in order to override virtuals. From that, two questions:

Is there any way to make this work as I supposed it may have worked?
Why would the compiler accept this code as valid? As I see it there are now two start() functions with the exact same signature in C and yet the compiler seems fine with it and only calls A::start().

EDIT: A few precisions: 

The goal is to manipulate C objects through A pointers.
I'm currently using a simple function that just calls B::start(), I was specifically wondering if a using declaration could indeed "override" a virtual, and if not, how this was allowed to have both functions coexist.
I might have omitted a few things like virtual inheritance for simplicity.

A:

Is there any way to make this work as I supposed it may have worked?

You should override the member function and explicitly call B::start():
class C: public A, private B {
public:
    void start() override { B::start(); }
};

Why would the compiler accept this code as valid? As I see it there
  are now two start() functions with the exact same signature in C and
  yet the compiler seems fine with it and only calls A::start().

You are right, there are two member functions accessible in C (A::start() and B::start()). And in class C, without overriding start() or making the start() of any of the base classes visible by doing a using ...::start(), you will have ambiguity error when trying to call the member function using unqalified namelookup from an object of C. 
class A {
public:
    virtual void start() { std::cout << "From A\n"; }
};

class B {
public:
    void start() { std::cout << "From B\n"; }
};

class C: public A, private B {
};

int main(){
    A* a = new C();
    a->start();       //Ok, calls A::start()

    C* c = new C();
    c->start();       //Error, ambiguous         
}

To fix that, you will have to use the qualified name such as:
    C* c = new C();
    c->A::start();       //Ok, calls A::start()

Now, doing a using B::start() in class C simply declares the start() to refer to B::start() whenever such name is used from an object of C
class A {
public:
    virtual void start() { std::cout << "From A\n"; }
};

class B {
public:
    void start() { std::cout << "From B\n"; }
};

class C: public A, private B {
public:
     using B::start();
};

int main(){
    A* a = new C();
    a->start();       //Ok, calls A::start()

    C* c = new C();
    c->start();       //Ok, calls B::start()
}

using B::start makes the function void B::start() visible in C, it does not override it. To call make all the above unqualified member function call, to call B::start(), you should override the member function in C, and make it call B::start()
class A {
public:
    virtual void start() { std::cout << "From A\n"; }
};

class B {
public:
    void start() { std::cout << "From B\n"; }
};

class C: public A, private B {
public:
    void start() override { B::start(); }
};

int main(){
    A* a = new C();
    a->start();         //Ok, calls C::start() which in turn calls B::start()
                        //    ^^^^^^^^^^^^^^^^ - by virtual dispatch

    C* c = new C();
    c->start();         //Ok, calls C::start() which in turn calls B::start()

}

